-- Logs begin at Wed 2022-08-31 00:32:43 UTC, end at Wed 2022-08-31 01:45:20 UTC. -- Aug 31 01:02:46.868667 dv-cinder-855288-2-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Aug 31 01:02:46.876609 dv-cinder-855288-2-iscsi devstack@keystone.service[81737]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Aug 31 01:02:46.878634 dv-cinder-855288-2-iscsi devstack@keystone.service[81737]: *** Starting uWSGI 2.0.18-debian (64bit) on [Wed Aug 31 01:02:46 2022] *** Aug 31 01:02:46.878634 dv-cinder-855288-2-iscsi devstack@keystone.service[81737]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 31 01:02:46.878634 dv-cinder-855288-2-iscsi devstack@keystone.service[81737]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 31 01:02:46.878634 dv-cinder-855288-2-iscsi devstack@keystone.service[81737]: nodename: dv-cinder-855288-2-iscsi Aug 31 01:02:46.878634 dv-cinder-855288-2-iscsi devstack@keystone.service[81737]: machine: x86_64 Aug 31 01:02:46.878634 dv-cinder-855288-2-iscsi devstack@keystone.service[81737]: clock source: unix Aug 31 01:02:46.878634 dv-cinder-855288-2-iscsi devstack@keystone.service[81737]: pcre jit disabled Aug 31 01:02:46.878634 dv-cinder-855288-2-iscsi devstack@keystone.service[81737]: detected number of CPU cores: 4 Aug 31 01:02:46.879224 dv-cinder-855288-2-iscsi devstack@keystone.service[81737]: current working directory: / Aug 31 01:02:46.879224 dv-cinder-855288-2-iscsi devstack@keystone.service[81737]: detected binary path: /usr/bin/uwsgi-core Aug 31 01:02:46.879224 dv-cinder-855288-2-iscsi devstack@keystone.service[81737]: your processes number limit is 47680 Aug 31 01:02:46.879224 dv-cinder-855288-2-iscsi devstack@keystone.service[81737]: your memory page size is 4096 bytes Aug 31 01:02:46.879224 dv-cinder-855288-2-iscsi devstack@keystone.service[81737]: detected max file descriptor number: 2048 Aug 31 01:02:46.879224 dv-cinder-855288-2-iscsi devstack@keystone.service[81737]: lock engine: pthread robust mutexes Aug 31 01:02:46.879224 dv-cinder-855288-2-iscsi devstack@keystone.service[81737]: thunder lock: enabled Aug 31 01:02:46.879224 dv-cinder-855288-2-iscsi devstack@keystone.service[81737]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Aug 31 01:02:46.879224 dv-cinder-855288-2-iscsi devstack@keystone.service[81737]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 31 01:02:46.919427 dv-cinder-855288-2-iscsi devstack@keystone.service[81737]: Python main interpreter initialized at 0x556a29e8e9c0 Aug 31 01:02:46.919427 dv-cinder-855288-2-iscsi devstack@keystone.service[81737]: python threads support enabled Aug 31 01:02:46.919427 dv-cinder-855288-2-iscsi devstack@keystone.service[81737]: your server socket listen backlog is limited to 100 connections Aug 31 01:02:46.919427 dv-cinder-855288-2-iscsi devstack@keystone.service[81737]: your mercy for graceful operations on workers is 90 seconds Aug 31 01:02:46.919779 dv-cinder-855288-2-iscsi devstack@keystone.service[81737]: mapped 403077 bytes (393 KB) for 2 cores Aug 31 01:02:46.919905 dv-cinder-855288-2-iscsi devstack@keystone.service[81737]: *** Operational MODE: preforking *** Aug 31 01:02:46.920191 dv-cinder-855288-2-iscsi devstack@keystone.service[81737]: *** uWSGI is running in multiple interpreter mode *** Aug 31 01:02:46.920191 dv-cinder-855288-2-iscsi devstack@keystone.service[81737]: spawned uWSGI master process (pid: 81737) Aug 31 01:02:46.920382 dv-cinder-855288-2-iscsi systemd[1]: Started Devstack devstack@keystone.service. Aug 31 01:02:46.922451 dv-cinder-855288-2-iscsi devstack@keystone.service[81737]: spawned uWSGI worker 1 (pid: 81749, cores: 1) Aug 31 01:02:46.922451 dv-cinder-855288-2-iscsi devstack@keystone.service[81737]: spawned uWSGI worker 2 (pid: 81750, cores: 1) Aug 31 01:02:46.922451 dv-cinder-855288-2-iscsi devstack@keystone.service[81737]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 31 01:02:51.263310 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 31 01:02:51.263310 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: from cryptography.utils import int_from_bytes Aug 31 01:02:51.264102 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 31 01:02:51.264102 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: from cryptography.utils import int_from_bytes Aug 31 01:02:51.268149 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 31 01:02:51.268149 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: from cryptography.utils import int_from_bytes Aug 31 01:02:51.268601 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 31 01:02:51.268601 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: from cryptography.utils import int_from_bytes Aug 31 01:02:51.314226 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 31 01:02:51.314416 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 31 01:02:51.314608 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] command line args: [] {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 31 01:02:51.314877 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 31 01:02:51.315057 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] ================================================================================ {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 31 01:02:51.315312 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] admin_token = **** {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 01:02:51.315560 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] config_dir = [] {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 01:02:51.315748 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 01:02:51.315824 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 31 01:02:51.315931 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] config_source = [] {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 01:02:51.316016 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 31 01:02:51.316114 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] debug = True {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 01:02:51.316192 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] command line args: [] {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 31 01:02:51.316316 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 01:02:51.316572 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 31 01:02:51.316572 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] ================================================================================ {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 31 01:02:51.316698 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 01:02:51.316866 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] admin_token = **** {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 01:02:51.316944 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 01:02:51.317094 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] config_dir = [] {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 01:02:51.317178 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 01:02:51.317284 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 01:02:51.317392 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 01:02:51.317487 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] config_source = [] {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 01:02:51.317590 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] list_limit = None {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 01:02:51.317690 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] debug = True {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 01:02:51.317784 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] log_config_append = None {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 01:02:51.317894 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 01:02:51.318022 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 01:02:51.318118 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 01:02:51.318225 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] log_dir = None {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 01:02:51.318320 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 01:02:51.318423 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] log_file = None {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 01:02:51.318523 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 01:02:51.318624 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 01:02:51.318701 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 01:02:51.318822 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 01:02:51.318903 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] list_limit = None {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 01:02:51.319028 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 01:02:51.319109 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] log_config_append = None {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 01:02:51.319186 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 01:02:51.319306 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 01:02:51.319380 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 01:02:51.319496 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] log_dir = None {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 01:02:51.319579 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 01:02:51.319657 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] log_file = None {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 01:02:51.319778 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 01:02:51.319858 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 01:02:51.319977 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 01:02:51.320065 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 01:02:51.320190 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 01:02:51.320267 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 01:02:51.320423 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 01:02:51.320540 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 01:02:51.320653 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 01:02:51.320808 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 01:02:51.320932 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 01:02:51.321052 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 01:02:51.321165 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 01:02:51.321282 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 01:02:51.321395 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 01:02:51.321518 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 01:02:51.321637 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 01:02:51.321764 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 01:02:51.321885 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 01:02:51.322003 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 01:02:51.322369 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.20/identity {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 01:02:51.322499 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 01:02:51.322618 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] publish_errors = False {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 01:02:51.322730 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 01:02:51.322849 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 01:02:51.322962 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 01:02:51.323072 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 01:02:51.323186 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 01:02:51.323303 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 01:02:51.323416 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 01:02:51.323607 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.20/identity {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 01:02:51.323734 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 01:02:51.323847 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] publish_errors = False {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 01:02:51.323958 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] standard_threads = False {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 01:02:51.324077 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 01:02:51.324204 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 01:02:51.324328 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 01:02:51.324448 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 01:02:51.324568 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 01:02:51.324692 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 01:02:51.324830 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 01:02:51.324954 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] use_journal = False {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 01:02:51.324954 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] use_json = False {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 01:02:51.325172 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 01:02:51.325284 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] use_stderr = False {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 01:02:51.325403 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] standard_threads = False {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 01:02:51.325522 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] use_syslog = False {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 01:02:51.325636 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 01:02:51.325754 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 01:02:51.325867 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 01:02:51.325984 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.326105 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 01:02:51.326105 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] use_journal = False {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 01:02:51.326314 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.326431 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] use_json = False {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 01:02:51.326549 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.326663 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] use_stderr = False {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 01:02:51.326845 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.326959 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] use_syslog = False {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 01:02:51.326959 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 01:02:51.327182 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.327295 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.327407 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.327532 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.327653 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.327765 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.327882 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.327994 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.328105 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.328224 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.328342 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.328454 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.328571 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.328683 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.328809 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.328933 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.329052 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.329165 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.329281 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.329393 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.329504 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.329630 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.329751 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.329869 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.329980 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.330091 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.330213 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.330326 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.330437 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.330553 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.330665 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.330782 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.330903 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.331022 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.331134 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.331245 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.331355 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.331472 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.331592 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.331705 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.331822 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.331935 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.332046 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.332157 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.332344 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.332464 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.332577 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.332695 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.332830 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.332944 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.333068 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.333182 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.333293 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.333404 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.333514 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.333631 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.333751 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.333863 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.333975 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.334086 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.334202 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.334319 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.334442 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.334555 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.334665 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.334776 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.334892 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.335006 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.335129 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.335243 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.335405 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.335523 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.335640 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.335738 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.335845 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.335947 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.336048 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.336148 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.336273 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.336381 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.336478 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.336577 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.336673 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.336821 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.336935 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.337059 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.337167 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.337278 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.337389 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.337502 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.337620 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.337743 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.337856 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.337968 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.338088 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.338690 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.338809 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.338922 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.339024 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.339144 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.339259 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.339369 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.339482 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.339594 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.339712 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.339833 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.339953 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.340074 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.340187 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.340300 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.340420 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.340535 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.340653 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.340808 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.340925 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.341038 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.341158 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.341280 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.341392 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.341495 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.341606 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.341718 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.341830 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.341950 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.342065 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.342176 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.342297 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.342418 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.342532 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.342656 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.342770 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.342888 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.343002 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.343113 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.343231 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.343352 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.343467 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.343578 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.343690 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.343809 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.343921 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.344109 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.344235 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.344350 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.344463 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.344575 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.344687 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.344868 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.344990 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.345104 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.345215 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.345327 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.345450 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.345574 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.345695 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.345810 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.345922 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.346034 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.346146 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.346264 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.346386 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.346501 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.346620 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.346733 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.346846 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.346958 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.347078 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.347192 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.347304 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.347424 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.347538 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.347652 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.347772 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.347886 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.347998 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.348110 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.348222 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.348334 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.348454 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.348568 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.348681 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.348818 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.348932 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.349054 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.349176 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.349291 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.349402 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.349514 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.349625 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.349738 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.349858 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.349972 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.350084 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.350196 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.350308 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.350420 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.350540 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.350654 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.350767 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.350879 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.350991 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.351103 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.351223 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.351338 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.351457 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.351571 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.351683 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] auth.external = None {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.351803 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.351925 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.352069 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.352196 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.352314 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.352427 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.352540 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.352653 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] auth.password = None {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.352814 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] auth.external = None {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.352814 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.353043 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] auth.token = None {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.353043 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.353255 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.353370 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.353491 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.353614 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.353729 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] auth.password = None {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.354556 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.354678 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] auth.token = None {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.354678 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.354902 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.355027 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.355144 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.355144 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.355357 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.355357 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.355578 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] credential.caching = True {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.355700 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.355816 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.355930 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.356044 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.356158 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.356299 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.356452 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] credential.caching = True {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.356571 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.356685 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.356825 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.356940 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.357068 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.357191 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.357306 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.357420 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.357533 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.357652 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.357775 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.357891 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.358025 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.358142 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.358257 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.358377 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.358567 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.358685 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.358799 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.358912 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.359025 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.359140 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.359261 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.359376 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.359489 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] federation.caching = True {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.359602 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.359716 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.359828 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.359949 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.360064 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.360179 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.360299 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] federation.caching = True {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.360413 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.360526 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.360647 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.360776 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.360892 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.361006 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.361119 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.361234 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.361360 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.361476 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.361590 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.361703 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.361816 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.361929 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.362050 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.362165 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.362280 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] identity.caching = True {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.362392 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.362512 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.362632 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.362754 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.362869 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.362982 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.363095 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] identity.caching = True {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.363213 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.363329 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.363451 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.363565 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.363679 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.363791 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.363904 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.364017 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.364138 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.364252 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.364373 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.364557 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.364677 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.364838 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.364967 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.365084 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.365197 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.365310 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.365489 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.365633 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.365759 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.365883 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.366005 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.366119 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.366233 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.366346 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.366469 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.366591 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.366591 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.366806 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.366919 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.367040 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.367156 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.367268 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.367268 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.367480 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.367594 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.367715 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.367830 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.367942 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.368055 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.368167 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.368280 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.368401 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.368516 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.368630 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.368630 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.368864 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.368864 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.369089 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.369089 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.369304 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.369417 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.369530 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.369644 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.369765 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.369880 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.369993 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.370106 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.370226 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.370340 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.370461 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.370575 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.370689 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.370793 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.370905 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.371017 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.371138 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.371252 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.371371 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.371485 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.371598 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.371712 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.371832 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.371947 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.372059 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.372173 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.372285 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.372398 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.372524 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.372640 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.372771 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.372887 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.373000 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.373112 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.373237 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.373354 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.373467 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.373588 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.373702 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.373817 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.373938 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.374059 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.374181 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.374295 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.374408 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] ldap.user = None {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.374524 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.374646 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.374761 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.374875 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.374988 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] ldap.user = None {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.375101 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.375213 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.375334 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.375449 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.375561 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.375674 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.375786 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.375899 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.376020 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.376134 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.376247 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.376359 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.376471 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.376584 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.376708 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.376857 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.376972 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.377085 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.377209 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.377324 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.377446 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.377561 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.377674 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.377787 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.377900 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.378014 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.378135 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.378250 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.378363 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.378475 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.378587 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.378700 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.378820 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.378935 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.379049 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.379162 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.379274 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.379395 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.379510 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.379623 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.379736 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.379849 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.379961 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.380083 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.380083 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.380297 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.380410 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.380522 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.380635 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.380839 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.380959 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.381080 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.381195 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.381308 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.381420 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.381542 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.381656 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.381769 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.381881 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.381994 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.382107 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.382229 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.382344 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.382457 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.382569 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.382682 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.382794 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.382915 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.383029 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.383142 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] resource.caching = True {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.383256 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.383376 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.383489 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] resource.caching = True {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.383617 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.383732 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.383846 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.383959 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.384072 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.384185 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.384307 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.384422 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.384538 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.384652 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.384777 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.384893 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.385505 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.385626 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.385740 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.385853 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.385966 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] role.caching = True {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.386078 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.386200 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] role.driver = sql {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.386314 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] role.caching = True {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.386434 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.386548 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] role.driver = sql {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.386661 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.386773 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.386895 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.387010 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.387123 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.387236 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.387351 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.387465 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.387586 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.387701 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.387815 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.387927 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.388040 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.388153 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.388274 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.388388 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.388502 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.388615 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.388781 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.388898 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.389023 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.389139 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.389252 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.389366 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.389480 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.389603 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.389603 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.389818 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.389818 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.390031 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.390143 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.390264 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.390379 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.390492 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.390606 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.390719 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.390838 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.390961 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.391076 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.391189 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.391303 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.391416 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.391529 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.391529 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.391752 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.391865 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.391977 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.392090 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.392203 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.392324 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.392438 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.392551 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.392665 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.392793 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.392907 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.393033 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.393148 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.393261 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.393374 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.393487 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.393599 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.393721 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.393836 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.393949 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] token.caching = True {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.394062 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.394175 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.394288 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.394409 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.394534 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] token.caching = True {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.394648 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.394762 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.394875 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.394988 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.395108 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.395223 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.395336 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.395448 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.395561 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.395673 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.395794 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.395910 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.396022 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.396135 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.396254 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.396368 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.396489 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.396604 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.396716 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.396899 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.397012 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.397125 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.397125 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.397353 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.397353 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.397574 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.397698 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.397813 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.397813 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.398043 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.398170 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.398285 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] database.connection = **** {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.398285 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.398497 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.398610 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.398722 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] database.connection = **** {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.398843 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.398959 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.399071 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.399184 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.399297 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.399409 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.399595 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.399712 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.399825 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.399938 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.400052 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.400164 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.400285 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.400401 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.400514 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.400627 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.401321 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.401439 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.401567 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.401684 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.401798 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.401912 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.402025 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.402139 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.402260 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.402375 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.402489 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.402602 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.402715 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.402829 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.402950 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.403065 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.403180 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 31 01:02:51.403294 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.403294 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 01:02:51.403294 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 31 01:02:51.430505 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81750) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 01:02:51.430633 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81749) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 01:02:51.430759 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81750) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 01:02:51.430838 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81749) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 01:02:51.430947 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81750) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 01:02:51.431048 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81749) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 01:02:51.439216 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81749) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 01:02:51.439305 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81750) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 01:02:51.468331 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81750) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 01:02:51.468528 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81749) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 01:02:51.468630 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81750) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 01:02:51.468779 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81749) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 01:02:51.521644 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81750) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 01:02:51.522017 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81749) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 01:02:51.570338 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81750) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 01:02:51.570569 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81750) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 01:02:51.570801 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81749) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 01:02:51.571037 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81749) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 01:02:51.727558 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81749) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 01:02:51.727795 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81749) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 01:02:51.728010 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81749) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 01:02:51.728238 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81749) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 01:02:51.728455 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81749) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 01:02:51.728602 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81750) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 01:02:51.728688 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81749) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 01:02:51.728918 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81750) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 01:02:51.729189 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81750) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 01:02:51.729420 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81750) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 01:02:51.729646 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81750) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 01:02:51.729869 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81750) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 01:02:51.751983 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81749) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 01:02:51.752215 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81749) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 01:02:51.752409 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81750) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 01:02:51.752493 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81749) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 01:02:51.752645 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81749) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 01:02:51.752756 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81750) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 01:02:51.752888 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81749) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 01:02:51.752984 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81750) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 01:02:51.753105 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81749) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 01:02:51.753188 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81750) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 01:02:51.753320 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81749) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 01:02:51.753406 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81750) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 01:02:51.753553 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81750) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 01:02:51.753774 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81750) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 01:02:51.764929 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81749) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 01:02:51.765508 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81750) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 01:02:51.777794 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81749) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 01:02:51.777982 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81749) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 01:02:51.778070 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81750) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 01:02:51.778205 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81749) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 01:02:51.778287 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81750) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 01:02:51.778423 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81749) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 01:02:51.778503 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81750) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 01:02:51.778638 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81749) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 01:02:51.778722 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81750) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 01:02:51.778854 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81749) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 01:02:51.778936 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81750) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 01:02:51.779065 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81749) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 01:02:51.779155 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81750) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 01:02:51.779346 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81750) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 01:02:51.795843 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 01:02:51.796049 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 31 01:02:51.796328 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81750) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 01:02:51.796425 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 01:02:51.796545 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81750) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 31 01:02:51.796627 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 31 01:02:51.796871 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 01:02:51.796955 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81750) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 01:02:51.797110 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81750) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 31 01:02:51.797205 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 01:02:51.797384 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 01:02:51.797466 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81750) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 01:02:51.797643 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 01:02:51.797723 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81750) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 01:02:51.798160 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81750) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 01:02:51.798433 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81750) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 01:02:51.807945 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 01:02:51.808221 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 01:02:51.808450 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81750) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 01:02:51.808534 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 01:02:51.808765 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81750) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 01:02:51.808852 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 01:02:51.809050 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81750) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 01:02:51.809342 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81750) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 01:02:51.815792 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81749) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 01:02:51.816361 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81750) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 01:02:51.818692 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81749) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 01:02:51.818980 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 01:02:51.819271 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 01:02:51.819354 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81750) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 01:02:51.819555 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 01:02:51.819638 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81750) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 01:02:51.819835 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 01:02:51.819916 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81750) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 01:02:51.820096 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 01:02:51.820180 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81750) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 01:02:51.820365 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 01:02:51.820448 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81750) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 01:02:51.820640 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 01:02:51.820738 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81750) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 01:02:51.820944 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81750) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 01:02:51.821041 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 01:02:51.821219 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 01:02:51.821298 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81750) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 01:02:51.821488 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 01:02:51.821609 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81750) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 01:02:51.821894 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81750) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 01:02:51.822439 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81750) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 01:02:51.845848 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81750) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 01:02:51.846570 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 01:02:51.847334 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81750) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 01:02:51.847624 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81750) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 01:02:51.848037 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81749) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 01:02:51.848325 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 01:02:51.852310 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81750) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 01:02:51.852621 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81750) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 01:02:51.852847 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81749) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 01:02:51.852944 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81750) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 01:02:51.853134 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 01:02:51.853416 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 01:02:51.859437 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81750) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 01:02:51.859730 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81750) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 01:02:51.859875 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81749) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 01:02:51.860166 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 01:02:51.863859 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81750) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 01:02:51.864102 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81749) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 01:02:51.864185 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81750) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 01:02:51.864392 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 01:02:51.864486 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81750) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 01:02:51.864678 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 01:02:51.864773 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81750) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 01:02:51.864986 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 01:02:51.865069 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81750) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 01:02:51.865265 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 01:02:51.865349 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81750) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 01:02:51.865541 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81750) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 01:02:51.865827 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81750) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 01:02:51.867189 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 01:02:51.867475 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 01:02:51.867759 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 01:02:51.883665 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 01:02:51.883767 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81750) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 01:02:51.883944 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 01:02:51.884029 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81750) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 01:02:51.888433 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81749) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 01:02:51.888527 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81750) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 01:02:51.891549 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 01:02:51.891724 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81750) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 01:02:51.891863 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 01:02:51.892012 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81750) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 01:02:51.896211 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81750) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 01:02:51.896379 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81749) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 01:02:51.899362 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81749) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 01:02:51.899458 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81750) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 01:02:51.902491 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 01:02:51.902640 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81750) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 01:02:51.902795 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 01:02:51.902948 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81750) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 01:02:51.903081 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 01:02:51.903243 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81750) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 01:02:51.903366 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 01:02:51.903534 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81750) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 01:02:51.903646 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 01:02:51.903830 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81750) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 01:02:51.904027 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 01:02:51.904109 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81750) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 01:02:51.919750 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81749) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 01:02:51.919968 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81750) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 01:02:51.920065 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 01:02:51.920263 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81750) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 01:02:51.920344 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 01:02:51.920535 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 01:02:51.920619 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81750) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 01:02:51.920824 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81750) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 01:02:51.927261 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 01:02:51.927713 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81750) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 01:02:51.928925 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 01:02:51.929283 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81750) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 01:02:51.930456 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 01:02:51.930734 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 01:02:51.930816 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81750) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 01:02:51.931105 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81750) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 01:02:51.933952 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81749) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 01:02:51.934240 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 01:02:51.934333 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81750) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 01:02:51.934533 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 01:02:51.934615 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81750) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 01:02:51.934816 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 01:02:51.934993 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81750) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 01:02:51.935107 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 01:02:51.935286 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81750) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 01:02:51.935581 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81750) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 01:02:51.946338 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81749) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 01:02:51.946625 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 01:02:51.946875 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81750) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 01:02:51.946971 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 01:02:51.947423 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81750) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 01:02:51.947526 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 01:02:51.948306 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 01:02:51.948392 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81750) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 01:02:51.948603 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 01:02:51.948687 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81750) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 01:02:51.948921 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 01:02:51.949017 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81750) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 01:02:51.949227 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81750) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 01:02:51.949536 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81750) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 01:02:51.964195 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81749) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 01:02:51.965078 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81750) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 01:02:51.967229 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81749) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 01:02:51.968002 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81750) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 01:02:51.970126 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 01:02:51.970914 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81750) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 01:02:51.971564 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 01:02:51.972362 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81750) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 01:02:51.973066 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81749) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 01:02:51.973362 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 01:02:51.973662 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 01:02:51.974011 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81750) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 01:02:51.974311 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81750) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 01:02:51.974615 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81750) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 01:02:51.980814 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 01:02:51.981097 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81750) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 01:02:51.982289 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81749) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 01:02:51.982588 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81750) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 01:02:51.985170 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 01:02:51.985475 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 01:02:51.985621 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81750) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 01:02:51.985766 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 01:02:51.985930 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81750) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 01:02:51.986048 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 01:02:51.986224 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81750) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 01:02:51.986511 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81750) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 01:02:51.993933 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81749) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 01:02:51.994228 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 01:02:51.994369 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81750) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 01:02:51.995043 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 01:02:51.995191 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81750) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 01:02:51.995479 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81750) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 01:02:52.001925 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81749) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 01:02:52.002213 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 01:02:52.002400 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81750) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 01:02:52.002495 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 01:02:52.002691 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81750) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 01:02:52.002775 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 01:02:52.002982 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81750) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 01:02:52.003065 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 01:02:52.003271 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81750) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 01:02:52.003363 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 01:02:52.003554 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 01:02:52.003636 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81750) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 01:02:52.003830 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 01:02:52.003912 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81750) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 01:02:52.004108 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 01:02:52.004191 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81750) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 01:02:52.004383 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 01:02:52.004464 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81750) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 01:02:52.004653 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 01:02:52.004827 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81750) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 01:02:52.004927 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 01:02:52.005001 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81750) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 01:02:52.005191 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 01:02:52.005273 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81750) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 01:02:52.005470 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 01:02:52.005625 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81750) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 01:02:52.005740 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81750) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 01:02:52.006028 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81750) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 01:02:52.039643 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x556a29e8e9c0 pid: 81749 (default app) Aug 31 01:02:52.040408 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x556a29e8e9c0 pid: 81750 (default app) Aug 31 01:02:52.043070 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0094091-c086-40c2-8a61-7ff6a03411d3 None None] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:02:52.043362 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0094091-c086-40c2-8a61-7ff6a03411d3 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:02:52.043635 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0094091-c086-40c2-8a61-7ff6a03411d3 None None] PATH_INFO: `/v3` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:02:52.047774 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1/1] 10.222.0.20 () {50 vars in 791 bytes} [Wed Aug 31 01:02:52 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:02:59.647033 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b546eb7d-884b-4ab4-9b9a-1482a8142723 None None] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:02:59.647600 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b546eb7d-884b-4ab4-9b9a-1482a8142723 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:02:59.647600 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b546eb7d-884b-4ab4-9b9a-1482a8142723 None None] PATH_INFO: `/` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:02:59.653131 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1/2] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:02:59 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:02:59.658519 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633b6e65-b09d-4209-a29c-bbc8d0524141 None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:02:59.658929 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633b6e65-b09d-4209-a29c-bbc8d0524141 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:02:59.659075 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633b6e65-b09d-4209-a29c-bbc8d0524141 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:02:59.729363 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: INFO dbcounter [None req-633b6e65-b09d-4209-a29c-bbc8d0524141 None None] Registered counter for database keystone Aug 31 01:02:59.750043 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG oslo_db.sqlalchemy.engines [None req-633b6e65-b09d-4209-a29c-bbc8d0524141 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81749) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 31 01:02:59.756240 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writer thread running {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 31 01:02:59.934860 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG passlib.handlers.bcrypt [None req-633b6e65-b09d-4209-a29c-bbc8d0524141 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81749) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 31 01:02:59.935246 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG passlib.handlers.bcrypt [None req-633b6e65-b09d-4209-a29c-bbc8d0524141 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81749) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 31 01:02:59.993682 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-633b6e65-b09d-4209-a29c-bbc8d0524141 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:02:59.995844 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-633b6e65-b09d-4209-a29c-bbc8d0524141 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:03:00.133299 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 2/3] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:02:59 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 476 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Aug 31 01:03:00.612637 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b546eb7d-884b-4ab4-9b9a-1482a8142723 None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:00.683337 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: INFO dbcounter [None req-b546eb7d-884b-4ab4-9b9a-1482a8142723 None None] Registered counter for database keystone Aug 31 01:03:00.702253 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG oslo_db.sqlalchemy.engines [None req-b546eb7d-884b-4ab4-9b9a-1482a8142723 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81750) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 31 01:03:00.708402 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writer thread running {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 31 01:03:00.827418 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b546eb7d-884b-4ab4-9b9a-1482a8142723 None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:00.831586 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f091b0-0fb8-4b4e-9aa2-84bd0e97a98e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:00.832212 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f091b0-0fb8-4b4e-9aa2-84bd0e97a98e admin admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:00.832500 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f091b0-0fb8-4b4e-9aa2-84bd0e97a98e admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:00.832810 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f091b0-0fb8-4b4e-9aa2-84bd0e97a98e admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:00.841454 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f091b0-0fb8-4b4e-9aa2-84bd0e97a98e admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:00.891595 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: WARNING keystone.common.rbac_enforcer.enforcer [None req-e4f091b0-0fb8-4b4e-9aa2-84bd0e97a98e admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 31 01:03:00.891886 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f091b0-0fb8-4b4e-9aa2-84bd0e97a98e admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:00.902255 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 2/4] 10.222.0.20 () {58 vars in 1324 bytes} [Wed Aug 31 01:03:00 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 31 01:03:00.908261 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633b6e65-b09d-4209-a29c-bbc8d0524141 None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:00.922980 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633b6e65-b09d-4209-a29c-bbc8d0524141 None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:00.927327 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b4e484-1934-40eb-ac7b-fad36f225987 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:00.928440 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b4e484-1934-40eb-ac7b-fad36f225987 admin admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:00.928751 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b4e484-1934-40eb-ac7b-fad36f225987 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:00.929026 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b4e484-1934-40eb-ac7b-fad36f225987 admin admin] PATH_INFO: `/v3/projects/77c8e2846a11431eb786ad3a5192ea92` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:00.930577 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b4e484-1934-40eb-ac7b-fad36f225987 admin admin] RBAC: Authorizing `identity:get_project(project_id=77c8e2846a11431eb786ad3a5192ea92)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:00.983570 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: WARNING keystone.common.rbac_enforcer.enforcer [None req-b3b4e484-1934-40eb-ac7b-fad36f225987 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 31 01:03:00.983919 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b4e484-1934-40eb-ac7b-fad36f225987 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:00.988458 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 3/5] 10.222.0.20 () {56 vars in 1207 bytes} [Wed Aug 31 01:03:00 2022] GET /identity/v3/projects/77c8e2846a11431eb786ad3a5192ea92 => generated 344 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:03:02.846145 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c058f27-22f7-4970-b399-2e1dc67aa43c None None] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:02.846682 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c058f27-22f7-4970-b399-2e1dc67aa43c None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:02.846682 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c058f27-22f7-4970-b399-2e1dc67aa43c None None] PATH_INFO: `/` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:02.850752 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 3/6] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:03:02 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:03:02.856711 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd40c29f-493d-4a44-840d-62eadf76e94d None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:02.857175 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd40c29f-493d-4a44-840d-62eadf76e94d None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:02.857269 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd40c29f-493d-4a44-840d-62eadf76e94d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:02.877791 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-bd40c29f-493d-4a44-840d-62eadf76e94d None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:03:02.879315 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-bd40c29f-493d-4a44-840d-62eadf76e94d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:03:02.887100 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 4/7] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:03:02 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 31 01:03:03.160812 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c058f27-22f7-4970-b399-2e1dc67aa43c None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:03.173320 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c058f27-22f7-4970-b399-2e1dc67aa43c None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:03.177459 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67e19f2-27a2-4b07-8648-c5900a53037e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:03.178052 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67e19f2-27a2-4b07-8648-c5900a53037e admin admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:03.178337 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67e19f2-27a2-4b07-8648-c5900a53037e admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:03.178609 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67e19f2-27a2-4b07-8648-c5900a53037e admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:03.186209 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67e19f2-27a2-4b07-8648-c5900a53037e admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:03.187972 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67e19f2-27a2-4b07-8648-c5900a53037e admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:03.196512 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 4/8] 10.222.0.20 () {58 vars in 1324 bytes} [Wed Aug 31 01:03:03 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 01:03:03.202152 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd40c29f-493d-4a44-840d-62eadf76e94d None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:03.211265 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd40c29f-493d-4a44-840d-62eadf76e94d None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:03.216091 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9713bbbf-384a-4813-9a42-3636fd6520c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:03.216715 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9713bbbf-384a-4813-9a42-3636fd6520c1 admin admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:03.217028 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9713bbbf-384a-4813-9a42-3636fd6520c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:03.217304 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9713bbbf-384a-4813-9a42-3636fd6520c1 admin admin] PATH_INFO: `/v3/users/7772468996d2491aa2e6bc53000efa74` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:03.218615 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9713bbbf-384a-4813-9a42-3636fd6520c1 admin admin] RBAC: Authorizing `identity:get_user(user_id=7772468996d2491aa2e6bc53000efa74)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:03.220438 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9713bbbf-384a-4813-9a42-3636fd6520c1 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:03.222009 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 5/9] 10.222.0.20 () {56 vars in 1198 bytes} [Wed Aug 31 01:03:03 2022] GET /identity/v3/users/7772468996d2491aa2e6bc53000efa74 => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:03:05.150799 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ceb1033-b782-4d71-90c3-62469816aab7 None None] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:05.151294 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ceb1033-b782-4d71-90c3-62469816aab7 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:05.151294 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ceb1033-b782-4d71-90c3-62469816aab7 None None] PATH_INFO: `/` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:05.152632 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 5/10] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:03:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:03:05.157734 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0155412-6fdb-42ca-99b3-c794a8cfc944 None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:05.158111 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0155412-6fdb-42ca-99b3-c794a8cfc944 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:05.158402 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0155412-6fdb-42ca-99b3-c794a8cfc944 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:05.176519 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-d0155412-6fdb-42ca-99b3-c794a8cfc944 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:03:05.177704 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-d0155412-6fdb-42ca-99b3-c794a8cfc944 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:03:05.219501 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36015d3-6d02-4dc2-9108-f8f77ef43e99 None None] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:05.219785 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36015d3-6d02-4dc2-9108-f8f77ef43e99 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:05.220060 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36015d3-6d02-4dc2-9108-f8f77ef43e99 None None] PATH_INFO: `/` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:05.221401 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 6/11] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:03:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 01:03:05.226362 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94940283-c205-4087-9e18-5d94aceacaa0 None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:05.226648 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94940283-c205-4087-9e18-5d94aceacaa0 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:05.226924 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94940283-c205-4087-9e18-5d94aceacaa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:05.268144 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG passlib.handlers.bcrypt [None req-94940283-c205-4087-9e18-5d94aceacaa0 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81750) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 31 01:03:05.272817 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG passlib.handlers.bcrypt [None req-94940283-c205-4087-9e18-5d94aceacaa0 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81750) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 31 01:03:05.272936 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 6/12] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:03:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 01:03:05.317843 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-94940283-c205-4087-9e18-5d94aceacaa0 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:03:05.319404 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-94940283-c205-4087-9e18-5d94aceacaa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:03:05.420248 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 7/13] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:03:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 01:03:05.535126 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0155412-6fdb-42ca-99b3-c794a8cfc944 None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:05.558948 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:05.639742 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0155412-6fdb-42ca-99b3-c794a8cfc944 None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:05.672582 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5003915-5dbc-4368-ac8b-d0716a0315e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:05.673202 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5003915-5dbc-4368-ac8b-d0716a0315e2 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:05.673492 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5003915-5dbc-4368-ac8b-d0716a0315e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:05.673778 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5003915-5dbc-4368-ac8b-d0716a0315e2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:05.684625 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5003915-5dbc-4368-ac8b-d0716a0315e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 01:03:05.687961 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94940283-c205-4087-9e18-5d94aceacaa0 None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:05.693608 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application [None req-e5003915-5dbc-4368-ac8b-d0716a0315e2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 01:03:05.693608 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:03:05.693608 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:03:05.693608 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:03:05.693608 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:03:05.693608 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:03:05.693608 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:03:05.693608 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:03:05.693608 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:03:05.693608 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:03:05.693608 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:03:05.693608 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:03:05.693608 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 01:03:05.693608 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 01:03:05.693608 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 01:03:05.693608 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 01:03:05.694675 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 01:03:05.694675 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 01:03:05.694675 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 01:03:05.694675 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 01:03:05.694675 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:03:05.694675 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:03:05.694675 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:03:05.694675 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:03:05.694675 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:03:05.694675 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:03:05.694675 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:03:05.694675 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application with Lock( Aug 31 01:03:05.694675 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:03:05.694675 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:03:05.694675 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:03:05.694675 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:03:05.695618 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:03:05.695618 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:03:05.695618 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:03:05.695618 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:03:05.695618 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 01:03:05.695618 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 01:03:05.695618 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 01:03:05.695618 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 01:03:05.695618 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 01:03:05.695618 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 01:03:05.695618 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 01:03:05.695618 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Aug 31 01:03:05.695618 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 7/14] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Aug 31 01:03:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:03:05.701079 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5003915-5dbc-4368-ac8b-d0716a0315e2 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:05.781794 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:05.789224 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94940283-c205-4087-9e18-5d94aceacaa0 None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:05.791487 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:05.799645 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5003915-5dbc-4368-ac8b-d0716a0315e2 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:05.826244 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a874eb-dcc8-4d45-a6d2-f446e33b71b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:05.827103 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a874eb-dcc8-4d45-a6d2-f446e33b71b4 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:05.827391 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a874eb-dcc8-4d45-a6d2-f446e33b71b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:05.827670 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a874eb-dcc8-4d45-a6d2-f446e33b71b4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:05.833685 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908b4e6d-a53d-4880-a5b0-e34ee2cbb9f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:05.834288 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908b4e6d-a53d-4880-a5b0-e34ee2cbb9f7 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:05.834571 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908b4e6d-a53d-4880-a5b0-e34ee2cbb9f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:05.834849 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908b4e6d-a53d-4880-a5b0-e34ee2cbb9f7 None admin] PATH_INFO: `/v3/roles` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:05.835515 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908b4e6d-a53d-4880-a5b0-e34ee2cbb9f7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:03:05.835821 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908b4e6d-a53d-4880-a5b0-e34ee2cbb9f7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:05.837267 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908b4e6d-a53d-4880-a5b0-e34ee2cbb9f7 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:05.849350 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 8/15] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Aug 31 01:03:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 01:03:05.854801 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908b4e6d-a53d-4880-a5b0-e34ee2cbb9f7 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:05.898220 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a874eb-dcc8-4d45-a6d2-f446e33b71b4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:05.899778 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a874eb-dcc8-4d45-a6d2-f446e33b71b4 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:05.952113 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908b4e6d-a53d-4880-a5b0-e34ee2cbb9f7 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:05.986106 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ffc029-fd50-4672-9d9e-f63fbdf77f47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:05.986732 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ffc029-fd50-4672-9d9e-f63fbdf77f47 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:05.987022 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ffc029-fd50-4672-9d9e-f63fbdf77f47 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:05.987301 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ffc029-fd50-4672-9d9e-f63fbdf77f47 None admin] PATH_INFO: `/v3/users/7772468996d2491aa2e6bc53000efa74` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:05.988455 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ffc029-fd50-4672-9d9e-f63fbdf77f47 None admin] RBAC: Authorizing `identity:get_user(user_id=7772468996d2491aa2e6bc53000efa74)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:05.989943 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ffc029-fd50-4672-9d9e-f63fbdf77f47 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:05.991511 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 9/16] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Aug 31 01:03:05 2022] GET /identity/v3/users/7772468996d2491aa2e6bc53000efa74 => generated 247 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 01:03:05.997495 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ffc029-fd50-4672-9d9e-f63fbdf77f47 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:05.997624 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 8/17] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Aug 31 01:03:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 31 01:03:06.003080 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a874eb-dcc8-4d45-a6d2-f446e33b71b4 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:06.005738 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:06.012821 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:06.095814 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ffc029-fd50-4672-9d9e-f63fbdf77f47 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:06.100741 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a874eb-dcc8-4d45-a6d2-f446e33b71b4 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:06.130504 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9e6e6a-30ab-4ded-b568-e67a718922f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:06.131138 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9e6e6a-30ab-4ded-b568-e67a718922f0 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:06.131422 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9e6e6a-30ab-4ded-b568-e67a718922f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:06.131698 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9e6e6a-30ab-4ded-b568-e67a718922f0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:06.132823 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9e6e6a-30ab-4ded-b568-e67a718922f0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:06.134281 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9e6e6a-30ab-4ded-b568-e67a718922f0 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:06.134375 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c2395a-a2f0-46d3-b49a-ae8e0e144a96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:06.135705 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c2395a-a2f0-46d3-b49a-ae8e0e144a96 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:06.135989 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c2395a-a2f0-46d3-b49a-ae8e0e144a96 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:06.136264 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c2395a-a2f0-46d3-b49a-ae8e0e144a96 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:06.136345 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 10/18] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 01:03:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 01:03:06.142481 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9e6e6a-30ab-4ded-b568-e67a718922f0 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:06.154622 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c2395a-a2f0-46d3-b49a-ae8e0e144a96 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:06.156086 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c2395a-a2f0-46d3-b49a-ae8e0e144a96 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:06.174073 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application [None req-43c2395a-a2f0-46d3-b49a-ae8e0e144a96 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 01:03:06.174073 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:03:06.174073 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:03:06.174073 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:03:06.174073 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:03:06.174073 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:03:06.174073 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:03:06.174073 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:03:06.174073 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:03:06.174073 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:03:06.174073 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:03:06.174073 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:03:06.174073 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 01:03:06.174073 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 01:03:06.174073 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 01:03:06.174073 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 01:03:06.174937 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:03:06.174937 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:03:06.174937 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:03:06.174937 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:03:06.174937 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:03:06.174937 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:03:06.174937 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:03:06.174937 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application with Lock( Aug 31 01:03:06.174937 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:03:06.174937 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:03:06.174937 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:03:06.174937 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:03:06.174937 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:03:06.174937 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:03:06.174937 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:03:06.174937 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:03:06.174937 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 01:03:06.180645 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 01:03:06.180645 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 01:03:06.180645 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Aug 31 01:03:06.180645 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 9/19] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 01:03:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 01:03:06.180900 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c2395a-a2f0-46d3-b49a-ae8e0e144a96 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:06.222168 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:06.237186 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:06.241145 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9e6e6a-30ab-4ded-b568-e67a718922f0 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:06.275934 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460b2192-cf5d-4b1b-a358-c41455bc9aad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:06.276573 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460b2192-cf5d-4b1b-a358-c41455bc9aad None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:06.276893 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460b2192-cf5d-4b1b-a358-c41455bc9aad None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:06.277173 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460b2192-cf5d-4b1b-a358-c41455bc9aad None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:06.277895 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460b2192-cf5d-4b1b-a358-c41455bc9aad None admin] RBAC: Adding query filter params (role.id=d009411496b8452b88e77ffb8db015a7, scope.domain.id=default, user.id=7772468996d2491aa2e6bc53000efa74) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:03:06.278205 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460b2192-cf5d-4b1b-a358-c41455bc9aad None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:06.279926 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460b2192-cf5d-4b1b-a358-c41455bc9aad None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:06.282686 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c2395a-a2f0-46d3-b49a-ae8e0e144a96 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:06.286528 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 11/20] 10.222.0.20 () {56 vars in 1322 bytes} [Wed Aug 31 01:03:06 2022] GET /identity/v3/role_assignments?user.id=7772468996d2491aa2e6bc53000efa74&scope.domain.id=default&role.id=d009411496b8452b88e77ffb8db015a7 => generated 233 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 01:03:06.318620 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb20e4b-df6c-43df-891c-59d973c36419 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:06.319256 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb20e4b-df6c-43df-891c-59d973c36419 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:06.319539 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb20e4b-df6c-43df-891c-59d973c36419 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:06.319814 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb20e4b-df6c-43df-891c-59d973c36419 None admin] PATH_INFO: `/v3/domains` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:06.320481 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb20e4b-df6c-43df-891c-59d973c36419 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:03:06.320823 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb20e4b-df6c-43df-891c-59d973c36419 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:06.322242 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb20e4b-df6c-43df-891c-59d973c36419 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:06.342431 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 10/21] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 31 01:03:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:03:08.462746 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01aee7c9-8d29-499d-b406-7c015022d80f None None] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:08.464135 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01aee7c9-8d29-499d-b406-7c015022d80f None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:08.464564 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01aee7c9-8d29-499d-b406-7c015022d80f None None] PATH_INFO: `/` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:08.466222 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 12/22] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:03:08 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:03:08.471717 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bf0b07-391a-42a8-9d0b-b3bcf3138242 None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:08.472532 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bf0b07-391a-42a8-9d0b-b3bcf3138242 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:08.473052 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bf0b07-391a-42a8-9d0b-b3bcf3138242 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:08.498405 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-37bf0b07-391a-42a8-9d0b-b3bcf3138242 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:03:08.499945 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-37bf0b07-391a-42a8-9d0b-b3bcf3138242 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:03:08.567907 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156225a2-6623-488f-bef1-dc4a138dd3a6 None None] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:08.568573 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156225a2-6623-488f-bef1-dc4a138dd3a6 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:08.569102 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156225a2-6623-488f-bef1-dc4a138dd3a6 None None] PATH_INFO: `/` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:08.570904 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 13/23] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:03:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 01:03:08.577395 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0544fba9-7c86-4401-a039-f2c2fc97a2f3 None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:08.577395 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0544fba9-7c86-4401-a039-f2c2fc97a2f3 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:08.577395 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0544fba9-7c86-4401-a039-f2c2fc97a2f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:08.596746 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 11/24] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:03:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 01:03:08.607023 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-0544fba9-7c86-4401-a039-f2c2fc97a2f3 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:03:08.612753 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-0544fba9-7c86-4401-a039-f2c2fc97a2f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:03:08.696188 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 14/25] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:03:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 01:03:08.982715 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bf0b07-391a-42a8-9d0b-b3bcf3138242 None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:09.044539 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:09.111350 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bf0b07-391a-42a8-9d0b-b3bcf3138242 None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:09.118677 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acaf3dd-74dc-4a8c-a783-e1538b0c1805 None None] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:09.118974 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acaf3dd-74dc-4a8c-a783-e1538b0c1805 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:09.119287 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acaf3dd-74dc-4a8c-a783-e1538b0c1805 None None] PATH_INFO: `/` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:09.120630 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 15/26] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:03:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:03:09.130843 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1458759b-51aa-4d5a-950f-fd1e4fa0195b None None] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:09.131394 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1458759b-51aa-4d5a-950f-fd1e4fa0195b None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:09.131874 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1458759b-51aa-4d5a-950f-fd1e4fa0195b None None] PATH_INFO: `/` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:09.133461 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 16/27] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:03:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 01:03:09.134711 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f51614b-40e2-4fb4-a938-3da2df27a4a8 None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:09.135245 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f51614b-40e2-4fb4-a938-3da2df27a4a8 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:09.135758 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f51614b-40e2-4fb4-a938-3da2df27a4a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:09.154646 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a266c138-3155-4b14-bba5-d86ce9a38c53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:09.155217 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a266c138-3155-4b14-bba5-d86ce9a38c53 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:09.155504 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a266c138-3155-4b14-bba5-d86ce9a38c53 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:09.155784 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a266c138-3155-4b14-bba5-d86ce9a38c53 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:09.167390 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-8f51614b-40e2-4fb4-a938-3da2df27a4a8 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:03:09.168170 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a266c138-3155-4b14-bba5-d86ce9a38c53 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 01:03:09.168613 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-8f51614b-40e2-4fb4-a938-3da2df27a4a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:03:09.175274 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application [None req-a266c138-3155-4b14-bba5-d86ce9a38c53 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 01:03:09.175274 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:03:09.175274 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:03:09.175274 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:03:09.175274 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:03:09.175274 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:03:09.175274 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:03:09.175274 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:03:09.175274 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:03:09.175274 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:03:09.175274 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:03:09.175274 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:03:09.175274 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 01:03:09.175274 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 01:03:09.175274 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 01:03:09.175274 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 01:03:09.176118 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 01:03:09.176118 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 01:03:09.176118 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 01:03:09.176118 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 01:03:09.176118 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:03:09.176118 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:03:09.176118 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:03:09.176118 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:03:09.176118 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:03:09.176118 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:03:09.176118 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:03:09.176118 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application with Lock( Aug 31 01:03:09.176118 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:03:09.176118 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:03:09.176118 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:03:09.176118 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:03:09.176901 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:03:09.176901 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:03:09.176901 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:03:09.176901 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:03:09.176901 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 01:03:09.176901 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 01:03:09.176901 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 01:03:09.176901 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 01:03:09.176901 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 01:03:09.176901 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 01:03:09.176901 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 01:03:09.176901 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Aug 31 01:03:09.176901 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 12/28] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Aug 31 01:03:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:03:09.178953 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0156c171-b635-48d4-9f01-bce84b387612 None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:09.179420 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0156c171-b635-48d4-9f01-bce84b387612 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:09.179842 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0156c171-b635-48d4-9f01-bce84b387612 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:09.200189 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-0156c171-b635-48d4-9f01-bce84b387612 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:03:09.201920 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-0156c171-b635-48d4-9f01-bce84b387612 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:03:09.266963 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 17/29] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:03:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 01:03:09.268385 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f51614b-40e2-4fb4-a938-3da2df27a4a8 None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:09.272649 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:09.291944 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 13/30] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:03:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 01:03:09.293282 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0156c171-b635-48d4-9f01-bce84b387612 None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:09.337378 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:09.417339 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0156c171-b635-48d4-9f01-bce84b387612 None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:09.464155 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef13f0f4-2854-4249-bac0-8b972a5e9a5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:09.464756 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f51614b-40e2-4fb4-a938-3da2df27a4a8 None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:09.465584 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef13f0f4-2854-4249-bac0-8b972a5e9a5a None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:09.466014 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef13f0f4-2854-4249-bac0-8b972a5e9a5a None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:09.466689 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef13f0f4-2854-4249-bac0-8b972a5e9a5a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:09.480164 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef13f0f4-2854-4249-bac0-8b972a5e9a5a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:09.482745 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef13f0f4-2854-4249-bac0-8b972a5e9a5a None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:09.499650 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application [None req-ef13f0f4-2854-4249-bac0-8b972a5e9a5a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 01:03:09.499650 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:03:09.499650 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:03:09.499650 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:03:09.499650 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:03:09.499650 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:03:09.499650 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:03:09.499650 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:03:09.499650 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:03:09.499650 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:03:09.499650 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:03:09.499650 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:03:09.499650 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 01:03:09.499650 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 01:03:09.499650 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 01:03:09.499650 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 01:03:09.500756 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:03:09.500756 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:03:09.500756 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:03:09.500756 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:03:09.500756 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:03:09.500756 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:03:09.500756 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:03:09.500756 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application with Lock( Aug 31 01:03:09.500756 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:03:09.500756 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:03:09.500756 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:03:09.500756 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:03:09.500756 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:03:09.500756 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:03:09.500756 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:03:09.500756 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:03:09.500756 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 01:03:09.501748 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 01:03:09.501748 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 01:03:09.501748 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Aug 31 01:03:09.503878 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 14/31] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 01:03:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:03:09.509092 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef13f0f4-2854-4249-bac0-8b972a5e9a5a None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:09.553428 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8bd3ce-24ac-4f75-829d-0b24b842f0c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:09.554054 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8bd3ce-24ac-4f75-829d-0b24b842f0c1 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:09.554337 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8bd3ce-24ac-4f75-829d-0b24b842f0c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:09.554610 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8bd3ce-24ac-4f75-829d-0b24b842f0c1 None admin] PATH_INFO: `/v3/roles` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:09.555468 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8bd3ce-24ac-4f75-829d-0b24b842f0c1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:03:09.555581 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8bd3ce-24ac-4f75-829d-0b24b842f0c1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:09.557018 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8bd3ce-24ac-4f75-829d-0b24b842f0c1 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:09.574276 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 18/32] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Aug 31 01:03:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:03:09.579458 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8bd3ce-24ac-4f75-829d-0b24b842f0c1 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:09.620316 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef13f0f4-2854-4249-bac0-8b972a5e9a5a None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:09.630290 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:09.657765 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59ff876-7304-4f37-8508-84e739298715 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:09.658728 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59ff876-7304-4f37-8508-84e739298715 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:09.659173 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59ff876-7304-4f37-8508-84e739298715 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:09.659628 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59ff876-7304-4f37-8508-84e739298715 None admin] PATH_INFO: `/v3/domains` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:09.660417 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59ff876-7304-4f37-8508-84e739298715 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:03:09.660890 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59ff876-7304-4f37-8508-84e739298715 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:09.662528 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59ff876-7304-4f37-8508-84e739298715 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:09.676263 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 15/33] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 31 01:03:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 01:03:09.682011 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59ff876-7304-4f37-8508-84e739298715 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:09.696502 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:09.747364 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8bd3ce-24ac-4f75-829d-0b24b842f0c1 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:09.793957 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c8e6ed-de67-4d67-ac3c-ff150bd88f48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:09.794786 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c8e6ed-de67-4d67-ac3c-ff150bd88f48 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:09.795168 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c8e6ed-de67-4d67-ac3c-ff150bd88f48 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:09.796161 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c8e6ed-de67-4d67-ac3c-ff150bd88f48 None admin] PATH_INFO: `/v3/users/7772468996d2491aa2e6bc53000efa74` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:09.797757 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c8e6ed-de67-4d67-ac3c-ff150bd88f48 None admin] RBAC: Authorizing `identity:get_user(user_id=7772468996d2491aa2e6bc53000efa74)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:09.800089 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c8e6ed-de67-4d67-ac3c-ff150bd88f48 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:09.802705 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 19/34] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Aug 31 01:03:09 2022] GET /identity/v3/users/7772468996d2491aa2e6bc53000efa74 => generated 247 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 01:03:09.803871 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c8e6ed-de67-4d67-ac3c-ff150bd88f48 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:09.816711 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59ff876-7304-4f37-8508-84e739298715 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:09.854332 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e8d220-36b2-4c41-bd75-c78a70200e46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:09.855149 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e8d220-36b2-4c41-bd75-c78a70200e46 None admin] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:09.855593 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e8d220-36b2-4c41-bd75-c78a70200e46 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:09.856027 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e8d220-36b2-4c41-bd75-c78a70200e46 None admin] PATH_INFO: `/v3/projects` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:09.857335 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e8d220-36b2-4c41-bd75-c78a70200e46 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:09.858722 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e8d220-36b2-4c41-bd75-c78a70200e46 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:09.921029 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c8e6ed-de67-4d67-ac3c-ff150bd88f48 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:09.957226 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba2a4ac-2b48-4cfb-a640-60a3918502b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:09.957771 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba2a4ac-2b48-4cfb-a640-60a3918502b8 None admin] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:09.958056 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba2a4ac-2b48-4cfb-a640-60a3918502b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:09.958338 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba2a4ac-2b48-4cfb-a640-60a3918502b8 None admin] PATH_INFO: `/v3/roles` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:09.959014 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba2a4ac-2b48-4cfb-a640-60a3918502b8 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:09.960410 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba2a4ac-2b48-4cfb-a640-60a3918502b8 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:09.964193 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:09.981997 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:10.104211 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 16/35] 10.222.0.20 () {60 vars in 1141 bytes} [Wed Aug 31 01:03:09 2022] POST /identity/v3/projects => generated 301 bytes in 417 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:03:10.104211 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e8d220-36b2-4c41-bd75-c78a70200e46 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:10.176529 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 20/36] 10.222.0.20 () {60 vars in 1131 bytes} [Wed Aug 31 01:03:09 2022] POST /identity/v3/roles => generated 219 bytes in 373 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:03:10.177811 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba2a4ac-2b48-4cfb-a640-60a3918502b8 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:10.229481 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e8d220-36b2-4c41-bd75-c78a70200e46 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:10.284482 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05f95df-8282-4bcb-ba26-e3ab6038002c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:10.286910 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05f95df-8282-4bcb-ba26-e3ab6038002c None admin] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:10.290797 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05f95df-8282-4bcb-ba26-e3ab6038002c None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:10.292920 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05f95df-8282-4bcb-ba26-e3ab6038002c None admin] PATH_INFO: `/v3/roles` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:10.293824 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05f95df-8282-4bcb-ba26-e3ab6038002c None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:10.297362 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05f95df-8282-4bcb-ba26-e3ab6038002c None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:10.316165 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba2a4ac-2b48-4cfb-a640-60a3918502b8 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:10.360892 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6531b98-294a-453f-ad0f-98932a584d43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:10.361640 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6531b98-294a-453f-ad0f-98932a584d43 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:10.361935 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6531b98-294a-453f-ad0f-98932a584d43 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:10.362218 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6531b98-294a-453f-ad0f-98932a584d43 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:10.363541 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6531b98-294a-453f-ad0f-98932a584d43 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:10.365380 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6531b98-294a-453f-ad0f-98932a584d43 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:10.367131 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 21/37] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 01:03:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:03:10.368543 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71b6002-c977-41d1-8d8c-500221becfa1 None None] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:10.368873 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71b6002-c977-41d1-8d8c-500221becfa1 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:10.369941 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71b6002-c977-41d1-8d8c-500221becfa1 None None] PATH_INFO: `/` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:10.371334 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 22/38] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:03:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:03:10.373848 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71b6002-c977-41d1-8d8c-500221becfa1 None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:10.378870 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 17/39] 10.222.0.20 () {60 vars in 1131 bytes} [Wed Aug 31 01:03:10 2022] POST /identity/v3/roles => generated 225 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:03:10.381968 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409ceccb-4760-4367-b955-696bdb710639 None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:10.382281 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409ceccb-4760-4367-b955-696bdb710639 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:10.382616 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409ceccb-4760-4367-b955-696bdb710639 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:10.411367 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-409ceccb-4760-4367-b955-696bdb710639 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:03:10.413401 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-409ceccb-4760-4367-b955-696bdb710639 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:03:10.469781 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:10.504623 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:10.523766 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 18/40] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:03:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 01:03:10.526515 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098e2b58-eeff-4ed4-84d7-d83287a76319 None None] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:10.526811 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098e2b58-eeff-4ed4-84d7-d83287a76319 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:10.527091 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098e2b58-eeff-4ed4-84d7-d83287a76319 None None] PATH_INFO: `/` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:10.528655 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 19/41] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:03:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:03:10.534220 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931fec5d-c694-4abe-9045-c30d8001094b None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:10.534507 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931fec5d-c694-4abe-9045-c30d8001094b None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:10.534783 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931fec5d-c694-4abe-9045-c30d8001094b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:10.540115 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71b6002-c977-41d1-8d8c-500221becfa1 None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:10.558894 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-931fec5d-c694-4abe-9045-c30d8001094b None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:03:10.558894 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-931fec5d-c694-4abe-9045-c30d8001094b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:03:10.590642 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9583ca17-6b83-4594-8d6e-4b0727e44281 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:10.591405 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9583ca17-6b83-4594-8d6e-4b0727e44281 None admin] REQUEST_METHOD: `PUT` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:10.591607 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9583ca17-6b83-4594-8d6e-4b0727e44281 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:10.591897 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9583ca17-6b83-4594-8d6e-4b0727e44281 None admin] PATH_INFO: `/v3/domains/default/users/7772468996d2491aa2e6bc53000efa74/roles/d009411496b8452b88e77ffb8db015a7` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:10.594067 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9583ca17-6b83-4594-8d6e-4b0727e44281 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=7772468996d2491aa2e6bc53000efa74, role_id=d009411496b8452b88e77ffb8db015a7)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:10.595537 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9583ca17-6b83-4594-8d6e-4b0727e44281 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:10.644131 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 20/42] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:03:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 01:03:10.693182 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 23/43] 10.222.0.20 () {58 vars in 1361 bytes} [Wed Aug 31 01:03:10 2022] PUT /identity/v3/domains/default/users/7772468996d2491aa2e6bc53000efa74/roles/d009411496b8452b88e77ffb8db015a7 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 01:03:10.808302 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931fec5d-c694-4abe-9045-c30d8001094b None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:10.914126 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9583ca17-6b83-4594-8d6e-4b0727e44281 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:10.915040 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931fec5d-c694-4abe-9045-c30d8001094b None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:10.937373 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:10.952661 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da110a6e-88f8-4c2e-9b06-9684d526d42d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:10.953364 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da110a6e-88f8-4c2e-9b06-9684d526d42d None admin] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:10.959478 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da110a6e-88f8-4c2e-9b06-9684d526d42d None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:10.959478 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da110a6e-88f8-4c2e-9b06-9684d526d42d None admin] PATH_INFO: `/v3/roles` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:10.959478 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da110a6e-88f8-4c2e-9b06-9684d526d42d None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:10.959478 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da110a6e-88f8-4c2e-9b06-9684d526d42d None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:11.016420 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9583ca17-6b83-4594-8d6e-4b0727e44281 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:11.039312 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:11.047077 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 21/44] 10.222.0.20 () {60 vars in 1131 bytes} [Wed Aug 31 01:03:10 2022] POST /identity/v3/roles => generated 223 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:03:11.053803 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7014c584-6f53-4805-a8c5-8f28c303b8f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:11.053803 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7014c584-6f53-4805-a8c5-8f28c303b8f6 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:11.053803 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7014c584-6f53-4805-a8c5-8f28c303b8f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:11.053803 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7014c584-6f53-4805-a8c5-8f28c303b8f6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:11.055079 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7014c584-6f53-4805-a8c5-8f28c303b8f6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:11.056582 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7014c584-6f53-4805-a8c5-8f28c303b8f6 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:11.058688 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 24/45] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 01:03:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:03:11.064561 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da110a6e-88f8-4c2e-9b06-9684d526d42d None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:11.179889 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da110a6e-88f8-4c2e-9b06-9684d526d42d None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:11.219166 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662eae0f-be93-459c-bec5-1e1fa5b5adbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:11.219789 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662eae0f-be93-459c-bec5-1e1fa5b5adbf None admin] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:11.220074 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662eae0f-be93-459c-bec5-1e1fa5b5adbf None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:11.220351 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662eae0f-be93-459c-bec5-1e1fa5b5adbf None admin] PATH_INFO: `/v3/projects` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:11.221058 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662eae0f-be93-459c-bec5-1e1fa5b5adbf None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:11.222525 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662eae0f-be93-459c-bec5-1e1fa5b5adbf None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:11.311077 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 22/46] 10.222.0.20 () {60 vars in 1141 bytes} [Wed Aug 31 01:03:11 2022] POST /identity/v3/projects => generated 312 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:03:12.598357 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6225dab5-26a6-4554-a66f-a019c7f59ab8 None None] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:12.598871 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6225dab5-26a6-4554-a66f-a019c7f59ab8 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:12.598992 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6225dab5-26a6-4554-a66f-a019c7f59ab8 None None] PATH_INFO: `/` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:12.600247 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 25/47] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:03:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:03:12.605517 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45dd5a8-63a9-4149-bc95-5e9a17b8cdd5 None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:12.605908 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45dd5a8-63a9-4149-bc95-5e9a17b8cdd5 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:12.606080 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45dd5a8-63a9-4149-bc95-5e9a17b8cdd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:12.623676 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-f45dd5a8-63a9-4149-bc95-5e9a17b8cdd5 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:03:12.624888 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-f45dd5a8-63a9-4149-bc95-5e9a17b8cdd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:03:12.650133 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:12.689801 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 23/48] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:03:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 01:03:12.953856 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6225dab5-26a6-4554-a66f-a019c7f59ab8 None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:13.160230 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6225dab5-26a6-4554-a66f-a019c7f59ab8 None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:13.194005 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53961194-9e38-4c80-956f-c8ceabb9a430 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:13.194631 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53961194-9e38-4c80-956f-c8ceabb9a430 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:13.194913 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53961194-9e38-4c80-956f-c8ceabb9a430 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:13.195215 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53961194-9e38-4c80-956f-c8ceabb9a430 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:13.202368 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34eeff52-8e47-4086-86b1-e0de2848d1aa None None] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:13.202658 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34eeff52-8e47-4086-86b1-e0de2848d1aa None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:13.202937 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34eeff52-8e47-4086-86b1-e0de2848d1aa None None] PATH_INFO: `/` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:13.204269 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 24/49] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:03:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:03:13.206007 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53961194-9e38-4c80-956f-c8ceabb9a430 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 01:03:13.209227 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2154a1e9-b7d6-4f93-9429-cb7f893cd6c4 None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:13.209521 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2154a1e9-b7d6-4f93-9429-cb7f893cd6c4 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:13.209804 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2154a1e9-b7d6-4f93-9429-cb7f893cd6c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:13.210922 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application [None req-53961194-9e38-4c80-956f-c8ceabb9a430 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 01:03:13.210922 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:03:13.210922 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:03:13.210922 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:03:13.210922 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:03:13.210922 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:03:13.210922 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:03:13.210922 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:03:13.210922 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:03:13.210922 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:03:13.210922 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:03:13.210922 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:03:13.210922 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 01:03:13.210922 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 01:03:13.210922 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 01:03:13.210922 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 01:03:13.211827 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 01:03:13.211827 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 01:03:13.211827 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 01:03:13.211827 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 01:03:13.211827 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:03:13.211827 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:03:13.211827 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:03:13.211827 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:03:13.211827 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:03:13.211827 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:03:13.211827 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:03:13.211827 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application with Lock( Aug 31 01:03:13.211827 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:03:13.211827 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:03:13.211827 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:03:13.211827 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:03:13.212584 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:03:13.212584 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:03:13.212584 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:03:13.212584 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:03:13.212584 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 01:03:13.212584 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 01:03:13.212584 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 01:03:13.212584 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 01:03:13.212584 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 01:03:13.212584 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 01:03:13.212584 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 01:03:13.212584 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Aug 31 01:03:13.212584 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 26/50] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Aug 31 01:03:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:03:13.217994 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53961194-9e38-4c80-956f-c8ceabb9a430 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:13.228267 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-2154a1e9-b7d6-4f93-9429-cb7f893cd6c4 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:03:13.230025 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-2154a1e9-b7d6-4f93-9429-cb7f893cd6c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:03:13.267099 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:13.295462 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 25/51] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:03:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 01:03:13.317274 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53961194-9e38-4c80-956f-c8ceabb9a430 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:13.349754 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55820e5-c938-451b-bc45-03bfb03e50ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:13.350365 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55820e5-c938-451b-bc45-03bfb03e50ac None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:13.350651 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55820e5-c938-451b-bc45-03bfb03e50ac None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:13.350929 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55820e5-c938-451b-bc45-03bfb03e50ac None admin] PATH_INFO: `/v3/roles` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:13.351592 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55820e5-c938-451b-bc45-03bfb03e50ac None admin] RBAC: Adding query filter params (name=admin) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:03:13.351900 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55820e5-c938-451b-bc45-03bfb03e50ac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:13.353329 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55820e5-c938-451b-bc45-03bfb03e50ac None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:13.362226 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 27/52] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Aug 31 01:03:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 01:03:13.367743 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2154a1e9-b7d6-4f93-9429-cb7f893cd6c4 None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:13.559909 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:13.567373 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55820e5-c938-451b-bc45-03bfb03e50ac None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:13.568331 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2154a1e9-b7d6-4f93-9429-cb7f893cd6c4 None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:13.603045 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a17190-221b-4e79-a3cf-e060aafe8d30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:13.604254 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a17190-221b-4e79-a3cf-e060aafe8d30 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:13.604534 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a17190-221b-4e79-a3cf-e060aafe8d30 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:13.604844 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a17190-221b-4e79-a3cf-e060aafe8d30 None admin] PATH_INFO: `/v3/users/7772468996d2491aa2e6bc53000efa74` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:13.606206 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a17190-221b-4e79-a3cf-e060aafe8d30 None admin] RBAC: Authorizing `identity:get_user(user_id=7772468996d2491aa2e6bc53000efa74)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:13.607669 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a17190-221b-4e79-a3cf-e060aafe8d30 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:13.609277 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 26/53] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Aug 31 01:03:13 2022] GET /identity/v3/users/7772468996d2491aa2e6bc53000efa74 => generated 247 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:03:13.614708 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a17190-221b-4e79-a3cf-e060aafe8d30 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:13.669124 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55820e5-c938-451b-bc45-03bfb03e50ac None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:13.682004 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:13.702421 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d363c9-2828-47c4-a794-47608a504365 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:13.703100 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d363c9-2828-47c4-a794-47608a504365 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:13.703385 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d363c9-2828-47c4-a794-47608a504365 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:13.703662 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d363c9-2828-47c4-a794-47608a504365 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:13.704807 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d363c9-2828-47c4-a794-47608a504365 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:13.706269 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d363c9-2828-47c4-a794-47608a504365 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:13.707830 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 28/54] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 01:03:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 01:03:13.713409 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a17190-221b-4e79-a3cf-e060aafe8d30 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:13.713955 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d363c9-2828-47c4-a794-47608a504365 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:13.746420 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fe8843-383a-4275-b2e8-637fd4b1d89a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:13.747048 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fe8843-383a-4275-b2e8-637fd4b1d89a None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:13.747329 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fe8843-383a-4275-b2e8-637fd4b1d89a None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:13.747605 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fe8843-383a-4275-b2e8-637fd4b1d89a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:13.748748 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fe8843-383a-4275-b2e8-637fd4b1d89a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:13.750210 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fe8843-383a-4275-b2e8-637fd4b1d89a None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:13.751743 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 27/55] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 01:03:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 01:03:13.757555 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fe8843-383a-4275-b2e8-637fd4b1d89a None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:13.785498 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:13.813829 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d363c9-2828-47c4-a794-47608a504365 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:13.847299 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35484843-af15-4f44-9a97-fb836c650c02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:13.847912 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35484843-af15-4f44-9a97-fb836c650c02 None admin] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:13.848200 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35484843-af15-4f44-9a97-fb836c650c02 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:13.848481 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35484843-af15-4f44-9a97-fb836c650c02 None admin] PATH_INFO: `/v3/projects` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:13.849172 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35484843-af15-4f44-9a97-fb836c650c02 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:13.850626 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35484843-af15-4f44-9a97-fb836c650c02 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:13.857058 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fe8843-383a-4275-b2e8-637fd4b1d89a None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:13.889687 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93facf42-387d-4e92-97be-763af5b9d67a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:13.890286 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93facf42-387d-4e92-97be-763af5b9d67a None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:13.890565 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93facf42-387d-4e92-97be-763af5b9d67a None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:13.890839 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93facf42-387d-4e92-97be-763af5b9d67a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:13.891597 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93facf42-387d-4e92-97be-763af5b9d67a None admin] RBAC: Adding query filter params (role.id=d009411496b8452b88e77ffb8db015a7, scope.domain.id=default, user.id=7772468996d2491aa2e6bc53000efa74) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:03:13.891897 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93facf42-387d-4e92-97be-763af5b9d67a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:13.893387 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93facf42-387d-4e92-97be-763af5b9d67a None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:13.902652 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 28/56] 10.222.0.20 () {56 vars in 1322 bytes} [Wed Aug 31 01:03:13 2022] GET /identity/v3/role_assignments?user.id=7772468996d2491aa2e6bc53000efa74&scope.domain.id=default&role.id=d009411496b8452b88e77ffb8db015a7 => generated 530 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 01:03:13.984493 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 29/57] 10.222.0.20 () {60 vars in 1140 bytes} [Wed Aug 31 01:03:13 2022] POST /identity/v3/projects => generated 298 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 01:03:15.875483 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6d2471-b4c1-4402-9082-04c428f1aa9c None None] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:15.875999 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6d2471-b4c1-4402-9082-04c428f1aa9c None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:15.876083 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6d2471-b4c1-4402-9082-04c428f1aa9c None None] PATH_INFO: `/` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:15.877627 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 29/58] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:03:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:03:15.882811 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94ce351-a515-469d-9c36-4331666aedb6 None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:15.883212 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94ce351-a515-469d-9c36-4331666aedb6 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:15.883384 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94ce351-a515-469d-9c36-4331666aedb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:15.902851 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-d94ce351-a515-469d-9c36-4331666aedb6 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:03:15.904381 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-d94ce351-a515-469d-9c36-4331666aedb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:03:15.930467 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:15.970533 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 30/59] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:03:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 01:03:16.236216 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6d2471-b4c1-4402-9082-04c428f1aa9c None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:16.324972 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:16.334034 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6d2471-b4c1-4402-9082-04c428f1aa9c None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:16.366777 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da4b909-f6bb-4716-bfbd-9bd01fad2ec0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:16.367400 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da4b909-f6bb-4716-bfbd-9bd01fad2ec0 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:16.367683 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da4b909-f6bb-4716-bfbd-9bd01fad2ec0 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:16.367957 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da4b909-f6bb-4716-bfbd-9bd01fad2ec0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:16.369117 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da4b909-f6bb-4716-bfbd-9bd01fad2ec0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:16.370546 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da4b909-f6bb-4716-bfbd-9bd01fad2ec0 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:16.372073 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 30/60] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 01:03:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:03:16.377384 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94ce351-a515-469d-9c36-4331666aedb6 None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:16.472056 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94ce351-a515-469d-9c36-4331666aedb6 None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:16.504224 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d06db6b-4ef6-4073-9e26-0b4b043c0eb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:16.504779 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d06db6b-4ef6-4073-9e26-0b4b043c0eb7 None admin] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:16.505064 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d06db6b-4ef6-4073-9e26-0b4b043c0eb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:16.505338 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d06db6b-4ef6-4073-9e26-0b4b043c0eb7 None admin] PATH_INFO: `/v3/users` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:16.505977 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d06db6b-4ef6-4073-9e26-0b4b043c0eb7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:16.507413 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d06db6b-4ef6-4073-9e26-0b4b043c0eb7 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:16.624263 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 31/61] 10.222.0.20 () {60 vars in 1132 bytes} [Wed Aug 31 01:03:16 2022] POST /identity/v3/users => generated 275 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:03:19.663987 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78f37c4-d717-4ea8-bf36-3b20ace46fad None None] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:19.665026 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78f37c4-d717-4ea8-bf36-3b20ace46fad None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:19.665693 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78f37c4-d717-4ea8-bf36-3b20ace46fad None None] PATH_INFO: `/` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:19.667464 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 31/62] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:03:19 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:03:19.672857 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6243006f-6c83-4ea9-b331-7441d8352401 None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:19.673705 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6243006f-6c83-4ea9-b331-7441d8352401 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:19.674189 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6243006f-6c83-4ea9-b331-7441d8352401 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:19.696859 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8102d672-8603-49bd-8253-1549080fed7f None None] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:19.698416 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8102d672-8603-49bd-8253-1549080fed7f None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:19.698911 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8102d672-8603-49bd-8253-1549080fed7f None None] PATH_INFO: `/` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:19.700456 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 32/63] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:03:19 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 01:03:19.704660 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-6243006f-6c83-4ea9-b331-7441d8352401 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:03:19.706155 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-6243006f-6c83-4ea9-b331-7441d8352401 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:03:19.713886 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8316aa1b-d56b-4c5b-ba3d-883f91f4f3db None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:19.714473 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8316aa1b-d56b-4c5b-ba3d-883f91f4f3db None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:19.714956 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8316aa1b-d56b-4c5b-ba3d-883f91f4f3db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:19.739963 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-8316aa1b-d56b-4c5b-ba3d-883f91f4f3db None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:03:19.741187 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-8316aa1b-d56b-4c5b-ba3d-883f91f4f3db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:03:19.746595 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:19.847010 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 32/64] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:03:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 01:03:19.862988 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 33/65] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:03:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 01:03:19.954954 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53635ea1-3ace-4910-b973-574d8afb43db None None] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:19.955259 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53635ea1-3ace-4910-b973-574d8afb43db None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:19.955527 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53635ea1-3ace-4910-b973-574d8afb43db None None] PATH_INFO: `/` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:19.957071 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 33/66] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:03:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:03:19.974128 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdc66a9-6da4-4bc3-a99b-c9cea293e7d1 None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:19.974418 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdc66a9-6da4-4bc3-a99b-c9cea293e7d1 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:19.974701 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdc66a9-6da4-4bc3-a99b-c9cea293e7d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:20.001679 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-1bdc66a9-6da4-4bc3-a99b-c9cea293e7d1 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:03:20.002867 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-1bdc66a9-6da4-4bc3-a99b-c9cea293e7d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:03:20.091148 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 34/67] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:03:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 01:03:20.159717 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97a271e-d625-45ff-9f12-8707a4ea00c3 None None] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:20.160226 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97a271e-d625-45ff-9f12-8707a4ea00c3 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:20.160648 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97a271e-d625-45ff-9f12-8707a4ea00c3 None None] PATH_INFO: `/` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:20.162428 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 34/68] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:03:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:03:20.167709 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d259e6-7d20-4000-8e4d-d0d83c99a01d None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:20.167995 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d259e6-7d20-4000-8e4d-d0d83c99a01d None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:20.168271 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d259e6-7d20-4000-8e4d-d0d83c99a01d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:20.176767 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97a271e-d625-45ff-9f12-8707a4ea00c3 None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:20.192587 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-b5d259e6-7d20-4000-8e4d-d0d83c99a01d None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:03:20.193802 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-b5d259e6-7d20-4000-8e4d-d0d83c99a01d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:03:20.209002 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:20.266764 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 35/69] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:03:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 01:03:20.267981 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d259e6-7d20-4000-8e4d-d0d83c99a01d None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:20.312933 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97a271e-d625-45ff-9f12-8707a4ea00c3 None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:20.352433 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8e9052-5913-4dff-b72e-640c98efc94e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:20.353169 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8e9052-5913-4dff-b72e-640c98efc94e None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:20.353538 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8e9052-5913-4dff-b72e-640c98efc94e None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:20.354394 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8e9052-5913-4dff-b72e-640c98efc94e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:20.368742 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8e9052-5913-4dff-b72e-640c98efc94e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 01:03:20.376034 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application [None req-ad8e9052-5913-4dff-b72e-640c98efc94e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 01:03:20.376034 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:03:20.376034 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:03:20.376034 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:03:20.376034 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:03:20.376034 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:03:20.376034 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:03:20.376034 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:03:20.376034 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:03:20.376034 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:03:20.376034 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:03:20.376034 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:03:20.376034 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 01:03:20.376034 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 01:03:20.376034 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 01:03:20.376034 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 01:03:20.376926 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 01:03:20.376926 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 01:03:20.376926 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 01:03:20.376926 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 01:03:20.376926 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:03:20.376926 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:03:20.376926 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:03:20.376926 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:03:20.376926 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:03:20.376926 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:03:20.376926 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:03:20.376926 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application with Lock( Aug 31 01:03:20.376926 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:03:20.376926 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:03:20.376926 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:03:20.376926 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:03:20.378064 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:03:20.378064 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:03:20.378064 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:03:20.378064 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:03:20.378064 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 01:03:20.378064 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 01:03:20.378064 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 01:03:20.378064 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 01:03:20.378064 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 01:03:20.378064 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 01:03:20.378064 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 01:03:20.378064 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Aug 31 01:03:20.378064 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 35/70] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Aug 31 01:03:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 01:03:20.378897 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256a7270-ebfc-4a60-80cd-5376f6d8b731 None None] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:20.379144 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256a7270-ebfc-4a60-80cd-5376f6d8b731 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:20.379424 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256a7270-ebfc-4a60-80cd-5376f6d8b731 None None] PATH_INFO: `/` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:20.381827 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 36/71] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:03:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:03:20.392745 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256a7270-ebfc-4a60-80cd-5376f6d8b731 None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:20.399694 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:20.407402 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d259e6-7d20-4000-8e4d-d0d83c99a01d None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:20.453383 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0d8802-f00c-433f-bbd5-bfc05dc4f14d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:20.454489 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0d8802-f00c-433f-bbd5-bfc05dc4f14d None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:20.455029 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0d8802-f00c-433f-bbd5-bfc05dc4f14d None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:20.455456 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0d8802-f00c-433f-bbd5-bfc05dc4f14d None admin] PATH_INFO: `/v3/roles/member` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:20.469349 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0d8802-f00c-433f-bbd5-bfc05dc4f14d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 01:03:20.476029 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application [None req-1b0d8802-f00c-433f-bbd5-bfc05dc4f14d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 01:03:20.476029 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:03:20.476029 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:03:20.476029 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:03:20.476029 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:03:20.476029 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:03:20.476029 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:03:20.476029 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:03:20.476029 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:03:20.476029 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:03:20.476029 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:03:20.476029 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:03:20.476029 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 01:03:20.476029 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 01:03:20.476029 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 01:03:20.476029 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 01:03:20.477108 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 01:03:20.477108 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 01:03:20.477108 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 01:03:20.477108 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 01:03:20.477108 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:03:20.477108 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:03:20.477108 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:03:20.477108 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:03:20.477108 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:03:20.477108 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:03:20.477108 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:03:20.477108 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application with Lock( Aug 31 01:03:20.477108 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:03:20.477108 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:03:20.477108 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:03:20.477108 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:03:20.478080 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:03:20.478080 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:03:20.478080 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:03:20.478080 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:03:20.478080 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 01:03:20.478080 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 01:03:20.478080 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 01:03:20.478080 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 01:03:20.478080 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 01:03:20.478080 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 01:03:20.478080 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 01:03:20.478080 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Aug 31 01:03:20.480700 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 36/72] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Aug 31 01:03:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:03:20.482532 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d0dc96-2f97-44ed-bdf3-b6e29573bd7f None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:20.482954 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d0dc96-2f97-44ed-bdf3-b6e29573bd7f None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:20.483360 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d0dc96-2f97-44ed-bdf3-b6e29573bd7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:20.497009 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256a7270-ebfc-4a60-80cd-5376f6d8b731 None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:20.504112 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-85d0dc96-2f97-44ed-bdf3-b6e29573bd7f None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:03:20.505332 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-85d0dc96-2f97-44ed-bdf3-b6e29573bd7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:03:20.508826 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:20.534006 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e87c251-8973-4d6f-95e3-2f92ad8f8e4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:20.534733 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e87c251-8973-4d6f-95e3-2f92ad8f8e4e None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:20.534951 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e87c251-8973-4d6f-95e3-2f92ad8f8e4e None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:20.535234 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e87c251-8973-4d6f-95e3-2f92ad8f8e4e None admin] PATH_INFO: `/v3/roles` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:20.535910 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e87c251-8973-4d6f-95e3-2f92ad8f8e4e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:03:20.536220 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e87c251-8973-4d6f-95e3-2f92ad8f8e4e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:20.537700 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e87c251-8973-4d6f-95e3-2f92ad8f8e4e None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:20.546260 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 37/73] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Aug 31 01:03:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 01:03:20.547069 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e87c251-8973-4d6f-95e3-2f92ad8f8e4e None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:20.579421 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 37/74] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:03:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 01:03:20.580104 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d0dc96-2f97-44ed-bdf3-b6e29573bd7f None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:20.623136 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:20.657404 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e87c251-8973-4d6f-95e3-2f92ad8f8e4e None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:20.680412 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d0dc96-2f97-44ed-bdf3-b6e29573bd7f None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:20.692318 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83a93c5-e8c0-499e-8645-a19693cb3302 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:20.692985 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83a93c5-e8c0-499e-8645-a19693cb3302 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:20.693410 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83a93c5-e8c0-499e-8645-a19693cb3302 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:20.693692 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83a93c5-e8c0-499e-8645-a19693cb3302 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:20.705116 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83a93c5-e8c0-499e-8645-a19693cb3302 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 01:03:20.711671 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application [None req-b83a93c5-e8c0-499e-8645-a19693cb3302 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 01:03:20.711671 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:03:20.711671 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:03:20.711671 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:03:20.711671 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:03:20.711671 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:03:20.711671 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:03:20.711671 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:03:20.711671 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:03:20.711671 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:03:20.711671 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:03:20.711671 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:03:20.711671 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 01:03:20.711671 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 01:03:20.711671 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 01:03:20.711671 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 01:03:20.712525 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 01:03:20.712525 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 01:03:20.712525 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 01:03:20.712525 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 01:03:20.712525 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:03:20.712525 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:03:20.712525 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:03:20.712525 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:03:20.712525 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:03:20.712525 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:03:20.712525 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:03:20.712525 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application with Lock( Aug 31 01:03:20.712525 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:03:20.712525 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:03:20.712525 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:03:20.712525 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:03:20.713825 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:03:20.713825 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:03:20.713825 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:03:20.713825 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:03:20.713825 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 01:03:20.713825 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 01:03:20.713825 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 01:03:20.713825 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 01:03:20.713825 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 01:03:20.713825 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 01:03:20.713825 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 01:03:20.713825 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Aug 31 01:03:20.713825 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 38/75] 10.222.0.20 () {56 vars in 1114 bytes} [Wed Aug 31 01:03:20 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:03:20.713825 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83a93c5-e8c0-499e-8645-a19693cb3302 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:20.719586 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98411399-3d56-4bbc-a156-83ac8c660776 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:20.720154 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98411399-3d56-4bbc-a156-83ac8c660776 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:20.720444 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98411399-3d56-4bbc-a156-83ac8c660776 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:20.720742 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98411399-3d56-4bbc-a156-83ac8c660776 None admin] PATH_INFO: `/v3/roles` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:20.721420 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98411399-3d56-4bbc-a156-83ac8c660776 None admin] RBAC: Adding query filter params (name=member) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:03:20.721733 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98411399-3d56-4bbc-a156-83ac8c660776 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:20.723132 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98411399-3d56-4bbc-a156-83ac8c660776 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:20.735902 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 38/76] 10.222.0.20 () {56 vars in 1101 bytes} [Wed Aug 31 01:03:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:03:20.737669 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98411399-3d56-4bbc-a156-83ac8c660776 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:20.740361 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:20.823897 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83a93c5-e8c0-499e-8645-a19693cb3302 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:20.849163 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98411399-3d56-4bbc-a156-83ac8c660776 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:20.852689 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:20.858846 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe10fe8d-a08d-4b77-b01e-af0e9fd69c74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:20.859477 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe10fe8d-a08d-4b77-b01e-af0e9fd69c74 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:20.859763 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe10fe8d-a08d-4b77-b01e-af0e9fd69c74 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:20.860041 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe10fe8d-a08d-4b77-b01e-af0e9fd69c74 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:20.861482 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe10fe8d-a08d-4b77-b01e-af0e9fd69c74 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:20.862961 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe10fe8d-a08d-4b77-b01e-af0e9fd69c74 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:20.866244 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 39/77] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 01:03:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:03:20.866244 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe10fe8d-a08d-4b77-b01e-af0e9fd69c74 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:20.887044 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130bd289-5536-499b-981a-9a8a8f8fb118 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:20.887686 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130bd289-5536-499b-981a-9a8a8f8fb118 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:20.887969 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130bd289-5536-499b-981a-9a8a8f8fb118 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:20.888246 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130bd289-5536-499b-981a-9a8a8f8fb118 None admin] PATH_INFO: `/v3/users/7772468996d2491aa2e6bc53000efa74` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:20.889407 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130bd289-5536-499b-981a-9a8a8f8fb118 None admin] RBAC: Authorizing `identity:get_user(user_id=7772468996d2491aa2e6bc53000efa74)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:20.890861 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130bd289-5536-499b-981a-9a8a8f8fb118 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:20.892497 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 39/78] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Aug 31 01:03:20 2022] GET /identity/v3/users/7772468996d2491aa2e6bc53000efa74 => generated 247 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:03:20.892497 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130bd289-5536-499b-981a-9a8a8f8fb118 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:20.963537 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:20.976164 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe10fe8d-a08d-4b77-b01e-af0e9fd69c74 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:20.993373 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130bd289-5536-499b-981a-9a8a8f8fb118 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:21.006071 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd33266d-370c-4032-ad7e-d9fb5eac996e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:21.006686 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd33266d-370c-4032-ad7e-d9fb5eac996e None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:21.006970 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd33266d-370c-4032-ad7e-d9fb5eac996e None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:21.007250 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd33266d-370c-4032-ad7e-d9fb5eac996e None admin] PATH_INFO: `/v3/roles` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:21.007915 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd33266d-370c-4032-ad7e-d9fb5eac996e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:03:21.008229 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd33266d-370c-4032-ad7e-d9fb5eac996e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:21.009673 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd33266d-370c-4032-ad7e-d9fb5eac996e None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:21.017324 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 40/79] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Aug 31 01:03:20 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:03:21.021006 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd33266d-370c-4032-ad7e-d9fb5eac996e None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:21.029442 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d78247-803f-4906-972a-cb192d369f4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:21.030073 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d78247-803f-4906-972a-cb192d369f4b None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:21.030358 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d78247-803f-4906-972a-cb192d369f4b None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:21.030637 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d78247-803f-4906-972a-cb192d369f4b None admin] PATH_INFO: `/v3/users/1aaea1e3d940441eb7f58357392a5940` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:21.045784 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d78247-803f-4906-972a-cb192d369f4b None admin] RBAC: Authorizing `identity:get_user(user_id=1aaea1e3d940441eb7f58357392a5940)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:21.047250 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d78247-803f-4906-972a-cb192d369f4b None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:21.048897 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 40/80] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Aug 31 01:03:20 2022] GET /identity/v3/users/1aaea1e3d940441eb7f58357392a5940 => generated 275 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:03:21.049645 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d78247-803f-4906-972a-cb192d369f4b None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:21.082011 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:21.121573 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd33266d-370c-4032-ad7e-d9fb5eac996e None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:21.147207 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d78247-803f-4906-972a-cb192d369f4b None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:21.155832 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de66349-5879-458f-b716-d0e9830ee441 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:21.156463 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de66349-5879-458f-b716-d0e9830ee441 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:21.156767 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de66349-5879-458f-b716-d0e9830ee441 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:21.157046 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de66349-5879-458f-b716-d0e9830ee441 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:21.167733 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de66349-5879-458f-b716-d0e9830ee441 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 01:03:21.173007 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application [None req-9de66349-5879-458f-b716-d0e9830ee441 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 01:03:21.173007 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:03:21.173007 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:03:21.173007 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:03:21.173007 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:03:21.173007 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:03:21.173007 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:03:21.173007 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:03:21.173007 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:03:21.173007 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:03:21.173007 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:03:21.173007 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:03:21.173007 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 01:03:21.173007 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 01:03:21.173007 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 01:03:21.173007 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 01:03:21.173939 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 01:03:21.173939 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 01:03:21.173939 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 01:03:21.173939 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 01:03:21.173939 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:03:21.173939 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:03:21.173939 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:03:21.173939 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:03:21.173939 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:03:21.173939 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:03:21.173939 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:03:21.173939 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application with Lock( Aug 31 01:03:21.173939 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:03:21.173939 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:03:21.173939 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:03:21.173939 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:03:21.174707 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:03:21.174707 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:03:21.174707 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:03:21.174707 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:03:21.174707 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 01:03:21.174707 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 01:03:21.174707 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 01:03:21.174707 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 01:03:21.174707 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 01:03:21.174707 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 01:03:21.174707 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 01:03:21.174707 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Aug 31 01:03:21.175291 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:21.175357 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 41/81] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Aug 31 01:03:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:03:21.175357 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de66349-5879-458f-b716-d0e9830ee441 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:21.185214 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7356c84-9150-4b3a-8999-b9480eb84ba4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:21.185837 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7356c84-9150-4b3a-8999-b9480eb84ba4 None admin] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:21.187847 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7356c84-9150-4b3a-8999-b9480eb84ba4 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:21.187847 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7356c84-9150-4b3a-8999-b9480eb84ba4 None admin] PATH_INFO: `/v3/users` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:21.187847 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7356c84-9150-4b3a-8999-b9480eb84ba4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:21.188781 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7356c84-9150-4b3a-8999-b9480eb84ba4 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:21.271835 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de66349-5879-458f-b716-d0e9830ee441 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:21.275130 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 41/82] 10.222.0.20 () {60 vars in 1132 bytes} [Wed Aug 31 01:03:21 2022] POST /identity/v3/users => generated 289 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:03:21.276013 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7356c84-9150-4b3a-8999-b9480eb84ba4 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:21.303947 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:21.320048 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467e52c9-9c37-43fc-8499-fcc32457936d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:21.320741 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467e52c9-9c37-43fc-8499-fcc32457936d None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:21.321031 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467e52c9-9c37-43fc-8499-fcc32457936d None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:21.321333 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467e52c9-9c37-43fc-8499-fcc32457936d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:21.395911 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7356c84-9150-4b3a-8999-b9480eb84ba4 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:21.400650 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467e52c9-9c37-43fc-8499-fcc32457936d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:21.402220 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467e52c9-9c37-43fc-8499-fcc32457936d None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:21.435923 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692a98d6-99fa-4ae3-bd0b-376547c43efb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:21.436905 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692a98d6-99fa-4ae3-bd0b-376547c43efb None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:21.436905 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692a98d6-99fa-4ae3-bd0b-376547c43efb None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:21.437267 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692a98d6-99fa-4ae3-bd0b-376547c43efb None admin] PATH_INFO: `/v3/users/1aaea1e3d940441eb7f58357392a5940` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:21.438437 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692a98d6-99fa-4ae3-bd0b-376547c43efb None admin] RBAC: Authorizing `identity:get_user(user_id=1aaea1e3d940441eb7f58357392a5940)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:21.439970 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692a98d6-99fa-4ae3-bd0b-376547c43efb None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:21.442535 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 42/83] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Aug 31 01:03:21 2022] GET /identity/v3/users/1aaea1e3d940441eb7f58357392a5940 => generated 275 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:03:21.442535 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692a98d6-99fa-4ae3-bd0b-376547c43efb None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:21.502452 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:21.523586 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 42/84] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Aug 31 01:03:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 01:03:21.524753 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467e52c9-9c37-43fc-8499-fcc32457936d None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:21.557340 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:21.568865 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692a98d6-99fa-4ae3-bd0b-376547c43efb None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:21.615853 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e83f25-3344-43e1-af99-4605c89bda5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:21.616479 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e83f25-3344-43e1-af99-4605c89bda5c None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:21.617519 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e83f25-3344-43e1-af99-4605c89bda5c None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:21.617519 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e83f25-3344-43e1-af99-4605c89bda5c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:21.666060 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467e52c9-9c37-43fc-8499-fcc32457936d None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:21.693327 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e83f25-3344-43e1-af99-4605c89bda5c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:21.694804 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e83f25-3344-43e1-af99-4605c89bda5c None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:21.712532 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9561cd-c31a-438c-87be-9c1d1a0b6833 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:21.714558 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9561cd-c31a-438c-87be-9c1d1a0b6833 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:21.715378 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9561cd-c31a-438c-87be-9c1d1a0b6833 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:21.715831 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9561cd-c31a-438c-87be-9c1d1a0b6833 None admin] PATH_INFO: `/v3/roles` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:21.716749 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9561cd-c31a-438c-87be-9c1d1a0b6833 None admin] RBAC: Adding query filter params (name=member) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:03:21.717243 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9561cd-c31a-438c-87be-9c1d1a0b6833 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:21.719399 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9561cd-c31a-438c-87be-9c1d1a0b6833 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:21.731137 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 43/85] 10.222.0.20 () {56 vars in 1101 bytes} [Wed Aug 31 01:03:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:03:21.733156 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9561cd-c31a-438c-87be-9c1d1a0b6833 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:21.750618 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:21.801585 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 43/86] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Aug 31 01:03:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 01:03:21.807656 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e83f25-3344-43e1-af99-4605c89bda5c None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:21.836863 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:21.859148 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9561cd-c31a-438c-87be-9c1d1a0b6833 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:21.902367 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf613bf9-241c-49fa-b32f-369c86d67a6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:21.903239 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf613bf9-241c-49fa-b32f-369c86d67a6d None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:21.903568 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf613bf9-241c-49fa-b32f-369c86d67a6d None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:21.903850 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf613bf9-241c-49fa-b32f-369c86d67a6d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:21.961244 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e83f25-3344-43e1-af99-4605c89bda5c None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:21.993367 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf613bf9-241c-49fa-b32f-369c86d67a6d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:21.995792 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf613bf9-241c-49fa-b32f-369c86d67a6d None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:22.007315 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69337a33-be69-4a67-9780-d2f70c61586c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:22.007980 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69337a33-be69-4a67-9780-d2f70c61586c None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:22.008638 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69337a33-be69-4a67-9780-d2f70c61586c None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:22.008949 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69337a33-be69-4a67-9780-d2f70c61586c None admin] PATH_INFO: `/v3/projects/82238b119c5840de9fc817856cec36fb` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:22.010949 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69337a33-be69-4a67-9780-d2f70c61586c None admin] RBAC: Authorizing `identity:get_project(project_id=82238b119c5840de9fc817856cec36fb)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:22.012712 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69337a33-be69-4a67-9780-d2f70c61586c None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:22.015276 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 44/87] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Aug 31 01:03:21 2022] GET /identity/v3/projects/82238b119c5840de9fc817856cec36fb => generated 298 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:03:22.015861 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69337a33-be69-4a67-9780-d2f70c61586c None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:22.038208 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:22.102767 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:22.121207 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69337a33-be69-4a67-9780-d2f70c61586c None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:22.126331 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 44/88] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Aug 31 01:03:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 01:03:22.127176 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf613bf9-241c-49fa-b32f-369c86d67a6d None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:22.166980 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c95392-316d-4719-9bd7-058e0bce1819 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:22.167614 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c95392-316d-4719-9bd7-058e0bce1819 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:22.167897 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c95392-316d-4719-9bd7-058e0bce1819 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:22.168173 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c95392-316d-4719-9bd7-058e0bce1819 None admin] PATH_INFO: `/v3/users/1aaea1e3d940441eb7f58357392a5940` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:22.169318 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c95392-316d-4719-9bd7-058e0bce1819 None admin] RBAC: Authorizing `identity:get_user(user_id=1aaea1e3d940441eb7f58357392a5940)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:22.171883 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c95392-316d-4719-9bd7-058e0bce1819 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:22.173652 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 45/89] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Aug 31 01:03:22 2022] GET /identity/v3/users/1aaea1e3d940441eb7f58357392a5940 => generated 275 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:03:22.174244 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c95392-316d-4719-9bd7-058e0bce1819 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:22.243863 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf613bf9-241c-49fa-b32f-369c86d67a6d None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:22.282121 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d2ad9a-42e5-465f-ab83-429665d44baa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:22.283358 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d2ad9a-42e5-465f-ab83-429665d44baa None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:22.284222 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d2ad9a-42e5-465f-ab83-429665d44baa None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:22.284699 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d2ad9a-42e5-465f-ab83-429665d44baa None admin] PATH_INFO: `/v3/projects/82238b119c5840de9fc817856cec36fb` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:22.286062 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d2ad9a-42e5-465f-ab83-429665d44baa None admin] RBAC: Authorizing `identity:get_project(project_id=82238b119c5840de9fc817856cec36fb)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:22.287203 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:22.290026 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d2ad9a-42e5-465f-ab83-429665d44baa None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:22.293659 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 45/90] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Aug 31 01:03:22 2022] GET /identity/v3/projects/82238b119c5840de9fc817856cec36fb => generated 298 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:03:22.295318 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d2ad9a-42e5-465f-ab83-429665d44baa None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:22.301366 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c95392-316d-4719-9bd7-058e0bce1819 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:22.347754 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5952fb30-4c6e-4b21-8b31-4fcb5baca160 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:22.348414 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5952fb30-4c6e-4b21-8b31-4fcb5baca160 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:22.348705 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5952fb30-4c6e-4b21-8b31-4fcb5baca160 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:22.349013 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5952fb30-4c6e-4b21-8b31-4fcb5baca160 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:22.350220 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5952fb30-4c6e-4b21-8b31-4fcb5baca160 None admin] RBAC: Adding query filter params (role.id=d009411496b8452b88e77ffb8db015a7, scope.project.id=82238b119c5840de9fc817856cec36fb, user.id=7772468996d2491aa2e6bc53000efa74) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:03:22.350537 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5952fb30-4c6e-4b21-8b31-4fcb5baca160 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:22.352574 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5952fb30-4c6e-4b21-8b31-4fcb5baca160 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:22.360516 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:22.361672 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 46/91] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Aug 31 01:03:22 2022] GET /identity/v3/role_assignments?user.id=7772468996d2491aa2e6bc53000efa74&scope.project.id=82238b119c5840de9fc817856cec36fb&role.id=d009411496b8452b88e77ffb8db015a7 => generated 259 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:03:22.362128 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5952fb30-4c6e-4b21-8b31-4fcb5baca160 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:22.483767 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d2ad9a-42e5-465f-ab83-429665d44baa None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:22.504189 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5952fb30-4c6e-4b21-8b31-4fcb5baca160 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:22.532254 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d469046-6cd4-429c-9cf9-ddbcb2683007 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:22.533087 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d469046-6cd4-429c-9cf9-ddbcb2683007 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:22.533383 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d469046-6cd4-429c-9cf9-ddbcb2683007 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:22.533691 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d469046-6cd4-429c-9cf9-ddbcb2683007 None admin] PATH_INFO: `/v3/projects/82238b119c5840de9fc817856cec36fb` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:22.535357 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d469046-6cd4-429c-9cf9-ddbcb2683007 None admin] RBAC: Authorizing `identity:get_project(project_id=82238b119c5840de9fc817856cec36fb)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:22.537168 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d469046-6cd4-429c-9cf9-ddbcb2683007 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:22.539440 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 46/92] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Aug 31 01:03:22 2022] GET /identity/v3/projects/82238b119c5840de9fc817856cec36fb => generated 298 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:03:22.540404 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d469046-6cd4-429c-9cf9-ddbcb2683007 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:22.548989 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc594ff5-9c2e-426b-a73c-c3a169f38b7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:22.549615 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc594ff5-9c2e-426b-a73c-c3a169f38b7d None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:22.549900 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc594ff5-9c2e-426b-a73c-c3a169f38b7d None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:22.550185 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc594ff5-9c2e-426b-a73c-c3a169f38b7d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:22.575437 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:22.662534 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d469046-6cd4-429c-9cf9-ddbcb2683007 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:22.682264 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:22.701199 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb4e5a8-caba-41d3-b0df-4ca2677a0806 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:22.701974 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb4e5a8-caba-41d3-b0df-4ca2677a0806 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:22.702270 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb4e5a8-caba-41d3-b0df-4ca2677a0806 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:22.702846 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb4e5a8-caba-41d3-b0df-4ca2677a0806 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:22.703774 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb4e5a8-caba-41d3-b0df-4ca2677a0806 None admin] RBAC: Adding query filter params (role.id=d0b9768f4d014766b9326387f2e11f34, scope.project.id=82238b119c5840de9fc817856cec36fb, user.id=1aaea1e3d940441eb7f58357392a5940) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:03:22.704115 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb4e5a8-caba-41d3-b0df-4ca2677a0806 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:22.705593 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb4e5a8-caba-41d3-b0df-4ca2677a0806 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:22.711658 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 47/93] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Aug 31 01:03:22 2022] GET /identity/v3/role_assignments?user.id=1aaea1e3d940441eb7f58357392a5940&scope.project.id=82238b119c5840de9fc817856cec36fb&role.id=d0b9768f4d014766b9326387f2e11f34 => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:03:22.713732 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb4e5a8-caba-41d3-b0df-4ca2677a0806 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:22.721162 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc594ff5-9c2e-426b-a73c-c3a169f38b7d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:22.722761 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc594ff5-9c2e-426b-a73c-c3a169f38b7d None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:22.832481 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb4e5a8-caba-41d3-b0df-4ca2677a0806 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:22.873907 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c077d8e0-27b7-4565-ab72-862fae1d7fa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:22.874538 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c077d8e0-27b7-4565-ab72-862fae1d7fa6 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:22.874831 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c077d8e0-27b7-4565-ab72-862fae1d7fa6 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:22.875121 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c077d8e0-27b7-4565-ab72-862fae1d7fa6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:22.875964 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c077d8e0-27b7-4565-ab72-862fae1d7fa6 None admin] RBAC: Adding query filter params (role.id=bebc77e8dfd54dbba7079f93218e7c7d, scope.project.id=82238b119c5840de9fc817856cec36fb, user.id=1aaea1e3d940441eb7f58357392a5940) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:03:22.876170 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c077d8e0-27b7-4565-ab72-862fae1d7fa6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:22.877673 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c077d8e0-27b7-4565-ab72-862fae1d7fa6 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:22.884238 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 48/94] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Aug 31 01:03:22 2022] GET /identity/v3/role_assignments?user.id=1aaea1e3d940441eb7f58357392a5940&scope.project.id=82238b119c5840de9fc817856cec36fb&role.id=bebc77e8dfd54dbba7079f93218e7c7d => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:03:22.976171 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 47/95] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Aug 31 01:03:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 614 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 01:03:22.982424 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c077d8e0-27b7-4565-ab72-862fae1d7fa6 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:23.051799 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:23.111165 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c077d8e0-27b7-4565-ab72-862fae1d7fa6 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:23.146572 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7461b9-1260-4318-aeb4-484bc4c9582a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:23.147247 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7461b9-1260-4318-aeb4-484bc4c9582a None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:23.147537 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7461b9-1260-4318-aeb4-484bc4c9582a None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:23.147823 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7461b9-1260-4318-aeb4-484bc4c9582a None admin] PATH_INFO: `/v3/projects/dc02a04f901440f2a30685de6c0a21d3` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:23.149015 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7461b9-1260-4318-aeb4-484bc4c9582a None admin] RBAC: Authorizing `identity:get_project(project_id=dc02a04f901440f2a30685de6c0a21d3)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:23.151041 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7461b9-1260-4318-aeb4-484bc4c9582a None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:23.152891 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 49/96] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Aug 31 01:03:22 2022] GET /identity/v3/projects/dc02a04f901440f2a30685de6c0a21d3 => generated 312 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:03:23.161846 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc594ff5-9c2e-426b-a73c-c3a169f38b7d None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:23.251763 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:23.314939 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc594ff5-9c2e-426b-a73c-c3a169f38b7d None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:23.359679 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3bab80-7f12-421b-81dd-cdc727e2f53d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:23.360298 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3bab80-7f12-421b-81dd-cdc727e2f53d None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:23.360590 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3bab80-7f12-421b-81dd-cdc727e2f53d None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:23.360897 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3bab80-7f12-421b-81dd-cdc727e2f53d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:23.361631 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3bab80-7f12-421b-81dd-cdc727e2f53d None admin] RBAC: Adding query filter params (role.id=d0b9768f4d014766b9326387f2e11f34, scope.project.id=dc02a04f901440f2a30685de6c0a21d3, user.id=1aaea1e3d940441eb7f58357392a5940) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:03:23.361948 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3bab80-7f12-421b-81dd-cdc727e2f53d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:23.363478 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3bab80-7f12-421b-81dd-cdc727e2f53d None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:23.373378 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 48/97] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Aug 31 01:03:23 2022] GET /identity/v3/role_assignments?user.id=1aaea1e3d940441eb7f58357392a5940&scope.project.id=dc02a04f901440f2a30685de6c0a21d3&role.id=d0b9768f4d014766b9326387f2e11f34 => generated 259 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:03:23.619255 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0a51dc-d7a0-4037-a134-597428642a25 None None] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:23.619587 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0a51dc-d7a0-4037-a134-597428642a25 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:23.619773 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0a51dc-d7a0-4037-a134-597428642a25 None None] PATH_INFO: `/` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:23.621250 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 50/98] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:03:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:03:23.626629 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a587273-a022-4e72-bff6-94f9ea41001e None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:23.627097 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a587273-a022-4e72-bff6-94f9ea41001e None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:23.627524 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a587273-a022-4e72-bff6-94f9ea41001e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:23.653726 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-0a587273-a022-4e72-bff6-94f9ea41001e None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:03:23.655164 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-0a587273-a022-4e72-bff6-94f9ea41001e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:03:23.727955 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13229afa-1c10-45d1-9a55-5f6482507040 None None] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:23.729802 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13229afa-1c10-45d1-9a55-5f6482507040 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:23.730563 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13229afa-1c10-45d1-9a55-5f6482507040 None None] PATH_INFO: `/` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:23.732604 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 51/99] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:03:23 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:03:23.736625 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 49/100] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:03:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 01:03:23.739733 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6990a9d0-e866-4360-94af-4056ce7af6fa None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:23.740227 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6990a9d0-e866-4360-94af-4056ce7af6fa None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:23.740700 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6990a9d0-e866-4360-94af-4056ce7af6fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:23.773747 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-6990a9d0-e866-4360-94af-4056ce7af6fa None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:03:23.775352 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-6990a9d0-e866-4360-94af-4056ce7af6fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:03:23.851395 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 52/101] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:03:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 01:03:24.138427 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a587273-a022-4e72-bff6-94f9ea41001e None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:24.199934 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:24.269497 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a587273-a022-4e72-bff6-94f9ea41001e None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:24.318138 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382346d6-c9e7-4769-9f1d-0e6a18def55b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:24.318811 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382346d6-c9e7-4769-9f1d-0e6a18def55b None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:24.319099 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382346d6-c9e7-4769-9f1d-0e6a18def55b None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:24.319375 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382346d6-c9e7-4769-9f1d-0e6a18def55b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:24.320535 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382346d6-c9e7-4769-9f1d-0e6a18def55b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:24.322137 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382346d6-c9e7-4769-9f1d-0e6a18def55b None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:24.324302 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 50/102] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 01:03:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:03:24.330062 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6990a9d0-e866-4360-94af-4056ce7af6fa None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:24.405344 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:24.431856 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382346d6-c9e7-4769-9f1d-0e6a18def55b None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:24.523467 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6990a9d0-e866-4360-94af-4056ce7af6fa None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:24.579200 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382346d6-c9e7-4769-9f1d-0e6a18def55b None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:24.619483 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220f247c-fcff-44b7-a74b-cd65e14f62c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:24.620112 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:24.620217 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220f247c-fcff-44b7-a74b-cd65e14f62c9 None admin] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:24.620418 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220f247c-fcff-44b7-a74b-cd65e14f62c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:24.620673 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220f247c-fcff-44b7-a74b-cd65e14f62c9 None admin] PATH_INFO: `/v3/projects` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:24.621422 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220f247c-fcff-44b7-a74b-cd65e14f62c9 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:24.622884 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220f247c-fcff-44b7-a74b-cd65e14f62c9 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:24.666204 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4258c04f-4941-454a-b810-b23456abd0d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:24.666850 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4258c04f-4941-454a-b810-b23456abd0d7 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:24.667136 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4258c04f-4941-454a-b810-b23456abd0d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:24.667415 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4258c04f-4941-454a-b810-b23456abd0d7 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:24.682290 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4258c04f-4941-454a-b810-b23456abd0d7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 01:03:24.687245 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application [None req-4258c04f-4941-454a-b810-b23456abd0d7 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 01:03:24.687245 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:03:24.687245 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:03:24.687245 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:03:24.687245 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:03:24.687245 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:03:24.687245 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:03:24.687245 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:03:24.687245 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:03:24.687245 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:03:24.687245 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:03:24.687245 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:03:24.687245 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 01:03:24.687245 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 01:03:24.687245 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 01:03:24.687245 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 01:03:24.688089 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 01:03:24.688089 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 01:03:24.688089 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 01:03:24.688089 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 01:03:24.688089 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:03:24.688089 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:03:24.688089 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:03:24.688089 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:03:24.688089 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:03:24.688089 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:03:24.688089 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:03:24.688089 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application with Lock( Aug 31 01:03:24.688089 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:03:24.688089 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:03:24.688089 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:03:24.688089 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:03:24.688883 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:03:24.688883 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:03:24.688883 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:03:24.688883 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:03:24.688883 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 01:03:24.688883 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 01:03:24.688883 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 01:03:24.688883 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 01:03:24.688883 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 01:03:24.688883 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 01:03:24.688883 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 01:03:24.688883 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Aug 31 01:03:24.688883 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 51/103] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Aug 31 01:03:24 2022] GET /identity/v3/roles/reader => generated 84 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:03:24.694510 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4258c04f-4941-454a-b810-b23456abd0d7 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:24.745425 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 53/104] 10.222.0.20 () {60 vars in 1141 bytes} [Wed Aug 31 01:03:24 2022] POST /identity/v3/projects => generated 302 bytes in 416 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:03:24.836399 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4258c04f-4941-454a-b810-b23456abd0d7 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:24.896875 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a425f448-6c09-41e2-a51c-dd37379ac29f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:24.897513 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a425f448-6c09-41e2-a51c-dd37379ac29f None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:24.897903 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a425f448-6c09-41e2-a51c-dd37379ac29f None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:24.898237 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a425f448-6c09-41e2-a51c-dd37379ac29f None admin] PATH_INFO: `/v3/roles` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:24.898937 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a425f448-6c09-41e2-a51c-dd37379ac29f None admin] RBAC: Adding query filter params (name=reader) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:03:24.899246 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a425f448-6c09-41e2-a51c-dd37379ac29f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:24.905313 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a425f448-6c09-41e2-a51c-dd37379ac29f None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:24.917261 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 52/105] 10.222.0.20 () {56 vars in 1101 bytes} [Wed Aug 31 01:03:24 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 01:03:24.925851 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220f247c-fcff-44b7-a74b-cd65e14f62c9 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:25.087380 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:25.093200 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220f247c-fcff-44b7-a74b-cd65e14f62c9 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:25.150835 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cf9ed6-d073-45f2-8f42-44c05a779938 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:25.151487 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cf9ed6-d073-45f2-8f42-44c05a779938 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:25.151776 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cf9ed6-d073-45f2-8f42-44c05a779938 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:25.152214 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cf9ed6-d073-45f2-8f42-44c05a779938 None admin] PATH_INFO: `/v3/users/855cb9a7a6794427b143420fd7d74353` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:25.171388 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cf9ed6-d073-45f2-8f42-44c05a779938 None admin] RBAC: Authorizing `identity:get_user(user_id=855cb9a7a6794427b143420fd7d74353)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:25.172917 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cf9ed6-d073-45f2-8f42-44c05a779938 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:25.174776 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 54/106] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Aug 31 01:03:24 2022] GET /identity/v3/users/855cb9a7a6794427b143420fd7d74353 => generated 289 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:03:25.180612 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a425f448-6c09-41e2-a51c-dd37379ac29f None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:25.244811 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:25.321075 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a425f448-6c09-41e2-a51c-dd37379ac29f None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:25.371736 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c322f8e7-def3-4ec2-86a6-435fff7c1f6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:25.372687 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c322f8e7-def3-4ec2-86a6-435fff7c1f6f None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:25.373189 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c322f8e7-def3-4ec2-86a6-435fff7c1f6f None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:25.373674 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c322f8e7-def3-4ec2-86a6-435fff7c1f6f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:25.436030 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dc62ec-c276-4e80-8a45-8be6efebceca None None] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:25.436344 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dc62ec-c276-4e80-8a45-8be6efebceca None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:25.436604 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dc62ec-c276-4e80-8a45-8be6efebceca None None] PATH_INFO: `/` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:25.438008 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 55/107] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:03:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:03:25.450186 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127c42eb-e68d-4332-9fa0-97ad8839b53f None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:25.450492 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127c42eb-e68d-4332-9fa0-97ad8839b53f None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:25.450760 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127c42eb-e68d-4332-9fa0-97ad8839b53f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:25.460280 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c322f8e7-def3-4ec2-86a6-435fff7c1f6f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:25.462040 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c322f8e7-def3-4ec2-86a6-435fff7c1f6f None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:25.491082 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-127c42eb-e68d-4332-9fa0-97ad8839b53f None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:03:25.492286 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-127c42eb-e68d-4332-9fa0-97ad8839b53f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:03:25.532302 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:25.616208 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 53/108] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Aug 31 01:03:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 01:03:25.617820 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02947c9d-b51e-4b51-a35d-2105eba0d383 None None] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:25.618114 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02947c9d-b51e-4b51-a35d-2105eba0d383 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:25.618397 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02947c9d-b51e-4b51-a35d-2105eba0d383 None None] PATH_INFO: `/` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:25.620437 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 54/109] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:03:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:03:25.626155 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01164a8a-b325-4068-900c-ef91ac005cae None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:25.626658 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01164a8a-b325-4068-900c-ef91ac005cae None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:25.627123 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01164a8a-b325-4068-900c-ef91ac005cae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:25.646443 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 56/110] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:03:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 01:03:25.647704 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127c42eb-e68d-4332-9fa0-97ad8839b53f None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:25.661201 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-01164a8a-b325-4068-900c-ef91ac005cae None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:03:25.664438 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-01164a8a-b325-4068-900c-ef91ac005cae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:03:25.757593 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:25.776984 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127c42eb-e68d-4332-9fa0-97ad8839b53f None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:25.818306 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 55/111] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:03:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 01:03:25.829663 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1feb147-c7a8-44c1-910e-4e95a4e16c8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:25.830598 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1feb147-c7a8-44c1-910e-4e95a4e16c8e None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:25.831073 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1feb147-c7a8-44c1-910e-4e95a4e16c8e None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:25.831537 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1feb147-c7a8-44c1-910e-4e95a4e16c8e None admin] PATH_INFO: `/v3/projects/82238b119c5840de9fc817856cec36fb` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:25.832912 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1feb147-c7a8-44c1-910e-4e95a4e16c8e None admin] RBAC: Authorizing `identity:get_project(project_id=82238b119c5840de9fc817856cec36fb)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:25.834767 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1feb147-c7a8-44c1-910e-4e95a4e16c8e None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:25.836604 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 57/112] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Aug 31 01:03:25 2022] GET /identity/v3/projects/82238b119c5840de9fc817856cec36fb => generated 298 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:03:25.848271 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01164a8a-b325-4068-900c-ef91ac005cae None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:25.970961 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:26.014630 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01164a8a-b325-4068-900c-ef91ac005cae None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:26.063050 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1935a9-e09d-4dfc-a20a-6d3e373e25cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:26.063666 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1935a9-e09d-4dfc-a20a-6d3e373e25cc None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:26.063951 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1935a9-e09d-4dfc-a20a-6d3e373e25cc None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:26.064230 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1935a9-e09d-4dfc-a20a-6d3e373e25cc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:26.064983 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1935a9-e09d-4dfc-a20a-6d3e373e25cc None admin] RBAC: Adding query filter params (role.id=e1aef2df18ab425a8b78190b0fbfe8ba, scope.project.id=82238b119c5840de9fc817856cec36fb, user.id=855cb9a7a6794427b143420fd7d74353) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:03:26.065297 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1935a9-e09d-4dfc-a20a-6d3e373e25cc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:26.066749 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1935a9-e09d-4dfc-a20a-6d3e373e25cc None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:26.077370 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 56/113] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Aug 31 01:03:25 2022] GET /identity/v3/role_assignments?user.id=855cb9a7a6794427b143420fd7d74353&scope.project.id=82238b119c5840de9fc817856cec36fb&role.id=e1aef2df18ab425a8b78190b0fbfe8ba => generated 259 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:03:26.081143 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1feb147-c7a8-44c1-910e-4e95a4e16c8e None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:26.125808 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b447c4a-f45f-401a-abb2-cfd04881b967 None None] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:26.126130 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b447c4a-f45f-401a-abb2-cfd04881b967 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:26.126380 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b447c4a-f45f-401a-abb2-cfd04881b967 None None] PATH_INFO: `/` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:26.127767 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 57/114] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:03:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:03:26.133606 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5fdd67-81bc-42be-bca3-24b68447327c None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:26.133926 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5fdd67-81bc-42be-bca3-24b68447327c None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:26.134179 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5fdd67-81bc-42be-bca3-24b68447327c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:26.159552 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-9c5fdd67-81bc-42be-bca3-24b68447327c None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:03:26.160692 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-9c5fdd67-81bc-42be-bca3-24b68447327c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:03:26.288151 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1feb147-c7a8-44c1-910e-4e95a4e16c8e None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:26.298970 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 58/115] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:03:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 01:03:26.300644 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5fdd67-81bc-42be-bca3-24b68447327c None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:26.344698 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9161baa-a014-4a48-88f3-4ca86edc4bb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:26.345359 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9161baa-a014-4a48-88f3-4ca86edc4bb3 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:26.345639 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9161baa-a014-4a48-88f3-4ca86edc4bb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:26.345912 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9161baa-a014-4a48-88f3-4ca86edc4bb3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:26.359765 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9161baa-a014-4a48-88f3-4ca86edc4bb3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 01:03:26.364950 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application [None req-d9161baa-a014-4a48-88f3-4ca86edc4bb3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 01:03:26.364950 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:03:26.364950 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:03:26.364950 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:03:26.364950 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:03:26.364950 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:03:26.364950 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:03:26.364950 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:03:26.364950 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:03:26.364950 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:03:26.364950 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:03:26.364950 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:03:26.364950 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 01:03:26.364950 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 01:03:26.364950 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 01:03:26.364950 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 01:03:26.366006 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 01:03:26.366006 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 01:03:26.366006 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 01:03:26.366006 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 01:03:26.366006 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:03:26.366006 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:03:26.366006 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:03:26.366006 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:03:26.366006 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:03:26.366006 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:03:26.366006 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:03:26.366006 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application with Lock( Aug 31 01:03:26.366006 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:03:26.366006 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:03:26.366006 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:03:26.366006 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:03:26.366964 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:03:26.366964 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:03:26.366964 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:03:26.366964 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:03:26.366964 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 01:03:26.366964 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 01:03:26.366964 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 01:03:26.366964 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 01:03:26.366964 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 01:03:26.366964 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 01:03:26.366964 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 01:03:26.366964 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Aug 31 01:03:26.368974 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 58/116] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Aug 31 01:03:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:03:26.370397 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdd13e0-dd8a-4f80-8312-58caa0008110 None None] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:26.370829 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdd13e0-dd8a-4f80-8312-58caa0008110 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:26.371249 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdd13e0-dd8a-4f80-8312-58caa0008110 None None] PATH_INFO: `/` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:26.372713 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 59/117] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:03:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:03:26.377623 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdd13e0-dd8a-4f80-8312-58caa0008110 None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:26.393593 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:26.397090 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:26.478973 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5fdd67-81bc-42be-bca3-24b68447327c None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:26.485291 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdd13e0-dd8a-4f80-8312-58caa0008110 None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:26.523291 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e607a2-833e-44fa-8870-3893e8dd20f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:26.523908 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e607a2-833e-44fa-8870-3893e8dd20f9 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:26.524191 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e607a2-833e-44fa-8870-3893e8dd20f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:26.524468 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e607a2-833e-44fa-8870-3893e8dd20f9 None admin] PATH_INFO: `/v3/roles` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:26.525200 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e607a2-833e-44fa-8870-3893e8dd20f9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:03:26.525510 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e607a2-833e-44fa-8870-3893e8dd20f9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:26.528748 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e607a2-833e-44fa-8870-3893e8dd20f9 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:26.535785 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 60/118] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Aug 31 01:03:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:03:26.536901 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caac832-5d0e-4824-9a3a-908cb3d39389 None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:26.537203 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caac832-5d0e-4824-9a3a-908cb3d39389 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:26.537487 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caac832-5d0e-4824-9a3a-908cb3d39389 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:26.555565 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-8caac832-5d0e-4824-9a3a-908cb3d39389 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:03:26.557317 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-8caac832-5d0e-4824-9a3a-908cb3d39389 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:03:26.570033 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc58464-418e-4348-9862-b3ea9a0c1cfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:26.570968 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc58464-418e-4348-9862-b3ea9a0c1cfc None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:26.571444 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc58464-418e-4348-9862-b3ea9a0c1cfc None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:26.571969 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc58464-418e-4348-9862-b3ea9a0c1cfc None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:26.611703 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc58464-418e-4348-9862-b3ea9a0c1cfc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 01:03:26.620335 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application [None req-fcc58464-418e-4348-9862-b3ea9a0c1cfc None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 01:03:26.620335 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:03:26.620335 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:03:26.620335 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:03:26.620335 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:03:26.620335 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:03:26.620335 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:03:26.620335 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:03:26.620335 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:03:26.620335 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:03:26.620335 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:03:26.620335 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:03:26.620335 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 01:03:26.620335 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 01:03:26.620335 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 01:03:26.620335 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 01:03:26.621213 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 01:03:26.621213 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 01:03:26.621213 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 01:03:26.621213 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 01:03:26.621213 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:03:26.621213 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:03:26.621213 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:03:26.621213 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:03:26.621213 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:03:26.621213 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:03:26.621213 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:03:26.621213 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application with Lock( Aug 31 01:03:26.621213 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:03:26.621213 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:03:26.621213 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:03:26.621213 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:03:26.621988 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:03:26.621988 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:03:26.621988 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:03:26.621988 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:03:26.621988 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 01:03:26.621988 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 01:03:26.621988 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 01:03:26.621988 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 01:03:26.621988 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 01:03:26.621988 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 01:03:26.621988 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 01:03:26.621988 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Aug 31 01:03:26.626144 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 59/119] 10.222.0.20 () {56 vars in 1114 bytes} [Wed Aug 31 01:03:26 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 326 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:03:26.627148 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc58464-418e-4348-9862-b3ea9a0c1cfc None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:26.628462 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 61/120] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:03:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 01:03:26.632962 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8caac832-5d0e-4824-9a3a-908cb3d39389 None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:26.640790 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:26.734024 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc58464-418e-4348-9862-b3ea9a0c1cfc None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:26.754913 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:26.760094 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8caac832-5d0e-4824-9a3a-908cb3d39389 None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:26.769119 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a9b537-f4a5-44fd-98b7-5dfe2f2a89fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:26.769904 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a9b537-f4a5-44fd-98b7-5dfe2f2a89fa None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:26.770281 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a9b537-f4a5-44fd-98b7-5dfe2f2a89fa None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:26.770653 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a9b537-f4a5-44fd-98b7-5dfe2f2a89fa None admin] PATH_INFO: `/v3/users/7772468996d2491aa2e6bc53000efa74` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:26.771979 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a9b537-f4a5-44fd-98b7-5dfe2f2a89fa None admin] RBAC: Authorizing `identity:get_user(user_id=7772468996d2491aa2e6bc53000efa74)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:26.773735 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a9b537-f4a5-44fd-98b7-5dfe2f2a89fa None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:26.775398 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 60/121] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Aug 31 01:03:26 2022] GET /identity/v3/users/7772468996d2491aa2e6bc53000efa74 => generated 247 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:03:26.776372 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a9b537-f4a5-44fd-98b7-5dfe2f2a89fa None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:26.801086 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c401c33-9a62-4bd9-9e32-5ee5b2ba3cd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:26.802090 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c401c33-9a62-4bd9-9e32-5ee5b2ba3cd7 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:26.802571 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c401c33-9a62-4bd9-9e32-5ee5b2ba3cd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:26.803042 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c401c33-9a62-4bd9-9e32-5ee5b2ba3cd7 None admin] PATH_INFO: `/v3/roles` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:26.803904 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c401c33-9a62-4bd9-9e32-5ee5b2ba3cd7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:03:26.804403 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c401c33-9a62-4bd9-9e32-5ee5b2ba3cd7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:26.806188 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c401c33-9a62-4bd9-9e32-5ee5b2ba3cd7 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:26.815709 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 62/122] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Aug 31 01:03:26 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:03:26.816865 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c401c33-9a62-4bd9-9e32-5ee5b2ba3cd7 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:26.933073 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:27.002393 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c401c33-9a62-4bd9-9e32-5ee5b2ba3cd7 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:27.021620 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a9b537-f4a5-44fd-98b7-5dfe2f2a89fa None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:27.049885 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071c8286-2824-41bb-b97d-265d40017fe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:27.050532 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071c8286-2824-41bb-b97d-265d40017fe3 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:27.050820 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071c8286-2824-41bb-b97d-265d40017fe3 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:27.051102 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071c8286-2824-41bb-b97d-265d40017fe3 None admin] PATH_INFO: `/v3/projects/82238b119c5840de9fc817856cec36fb` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:27.053901 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071c8286-2824-41bb-b97d-265d40017fe3 None admin] RBAC: Authorizing `identity:get_project(project_id=82238b119c5840de9fc817856cec36fb)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:27.055362 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071c8286-2824-41bb-b97d-265d40017fe3 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:27.058618 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 63/123] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Aug 31 01:03:26 2022] GET /identity/v3/projects/82238b119c5840de9fc817856cec36fb => generated 298 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:03:27.059600 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071c8286-2824-41bb-b97d-265d40017fe3 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:27.078094 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eced335f-03e6-4807-abba-40e8fdd4d338 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:27.079351 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eced335f-03e6-4807-abba-40e8fdd4d338 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:27.080005 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eced335f-03e6-4807-abba-40e8fdd4d338 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:27.080421 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eced335f-03e6-4807-abba-40e8fdd4d338 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:27.094811 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eced335f-03e6-4807-abba-40e8fdd4d338 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 01:03:27.100988 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application [None req-eced335f-03e6-4807-abba-40e8fdd4d338 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 01:03:27.100988 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:03:27.100988 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:03:27.100988 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:03:27.100988 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:03:27.100988 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:03:27.100988 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:03:27.100988 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:03:27.100988 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:03:27.100988 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:03:27.100988 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:03:27.100988 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:03:27.100988 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 01:03:27.100988 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 01:03:27.100988 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 01:03:27.100988 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 01:03:27.101877 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 01:03:27.101877 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 01:03:27.101877 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 01:03:27.101877 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 01:03:27.101877 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:03:27.101877 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:03:27.101877 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:03:27.101877 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:03:27.101877 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:03:27.101877 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:03:27.101877 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:03:27.101877 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application with Lock( Aug 31 01:03:27.101877 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:03:27.101877 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:03:27.101877 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:03:27.101877 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:03:27.102632 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:03:27.102632 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:03:27.102632 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:03:27.102632 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:03:27.102632 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 01:03:27.102632 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 01:03:27.102632 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 01:03:27.102632 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 01:03:27.102632 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 01:03:27.102632 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 01:03:27.102632 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 01:03:27.102632 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Aug 31 01:03:27.104995 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 61/124] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Aug 31 01:03:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 329 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:03:27.106305 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eced335f-03e6-4807-abba-40e8fdd4d338 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:27.167605 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071c8286-2824-41bb-b97d-265d40017fe3 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:27.168688 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:27.205163 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ec4934-a7a5-424d-92d2-30b537a47ffa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:27.205687 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ec4934-a7a5-424d-92d2-30b537a47ffa None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:27.206105 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ec4934-a7a5-424d-92d2-30b537a47ffa None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:27.206614 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ec4934-a7a5-424d-92d2-30b537a47ffa None admin] PATH_INFO: `/v3/users/1aaea1e3d940441eb7f58357392a5940` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:27.208005 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ec4934-a7a5-424d-92d2-30b537a47ffa None admin] RBAC: Authorizing `identity:get_user(user_id=1aaea1e3d940441eb7f58357392a5940)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:27.209605 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ec4934-a7a5-424d-92d2-30b537a47ffa None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:27.212337 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 64/125] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Aug 31 01:03:27 2022] GET /identity/v3/users/1aaea1e3d940441eb7f58357392a5940 => generated 275 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:03:27.213125 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ec4934-a7a5-424d-92d2-30b537a47ffa None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:27.225194 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:27.238340 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eced335f-03e6-4807-abba-40e8fdd4d338 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:27.274847 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eac4c79-ff8e-4c00-8f76-06808c147b56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:27.275500 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eac4c79-ff8e-4c00-8f76-06808c147b56 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:27.275881 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eac4c79-ff8e-4c00-8f76-06808c147b56 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:27.276051 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eac4c79-ff8e-4c00-8f76-06808c147b56 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:27.286657 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eac4c79-ff8e-4c00-8f76-06808c147b56 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 01:03:27.291875 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application [None req-0eac4c79-ff8e-4c00-8f76-06808c147b56 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 01:03:27.291875 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:03:27.291875 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:03:27.291875 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:03:27.291875 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:03:27.291875 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:03:27.291875 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:03:27.291875 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:03:27.291875 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:03:27.291875 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:03:27.291875 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:03:27.291875 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:03:27.291875 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 01:03:27.291875 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 01:03:27.291875 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 01:03:27.291875 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 01:03:27.292715 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 01:03:27.292715 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 01:03:27.292715 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 01:03:27.292715 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 01:03:27.292715 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:03:27.292715 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:03:27.292715 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:03:27.292715 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:03:27.292715 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:03:27.292715 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:03:27.292715 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:03:27.292715 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application with Lock( Aug 31 01:03:27.292715 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:03:27.292715 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:03:27.292715 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:03:27.292715 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:03:27.293501 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:03:27.293501 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:03:27.293501 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:03:27.293501 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:03:27.293501 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 01:03:27.293501 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 01:03:27.293501 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 01:03:27.293501 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 01:03:27.293501 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 01:03:27.293501 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 01:03:27.293501 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 01:03:27.293501 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Aug 31 01:03:27.293501 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 62/126] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Aug 31 01:03:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:03:27.294120 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eac4c79-ff8e-4c00-8f76-06808c147b56 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:27.345263 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ec4934-a7a5-424d-92d2-30b537a47ffa None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:27.392382 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8dd76e-eb28-4dd8-b2e0-73341613d047 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:27.393122 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8dd76e-eb28-4dd8-b2e0-73341613d047 None admin] REQUEST_METHOD: `PUT` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:27.393816 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8dd76e-eb28-4dd8-b2e0-73341613d047 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:27.394113 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8dd76e-eb28-4dd8-b2e0-73341613d047 None admin] PATH_INFO: `/v3/projects/82238b119c5840de9fc817856cec36fb/users/7772468996d2491aa2e6bc53000efa74/roles/d009411496b8452b88e77ffb8db015a7` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:27.396691 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8dd76e-eb28-4dd8-b2e0-73341613d047 None admin] RBAC: Authorizing `identity:create_grant(project_id=82238b119c5840de9fc817856cec36fb, user_id=7772468996d2491aa2e6bc53000efa74, role_id=d009411496b8452b88e77ffb8db015a7)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:27.398965 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8dd76e-eb28-4dd8-b2e0-73341613d047 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:27.416973 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eac4c79-ff8e-4c00-8f76-06808c147b56 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:27.426883 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:27.451718 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520bdf07-3040-4fa7-b498-66c3df39c6da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:27.452326 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520bdf07-3040-4fa7-b498-66c3df39c6da None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:27.452608 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520bdf07-3040-4fa7-b498-66c3df39c6da None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:27.452911 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520bdf07-3040-4fa7-b498-66c3df39c6da None admin] PATH_INFO: `/v3/roles` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:27.453569 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520bdf07-3040-4fa7-b498-66c3df39c6da None admin] RBAC: Adding query filter params (name=member) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:03:27.453871 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520bdf07-3040-4fa7-b498-66c3df39c6da None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:27.455250 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520bdf07-3040-4fa7-b498-66c3df39c6da None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:27.463160 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 63/127] 10.222.0.20 () {56 vars in 1101 bytes} [Wed Aug 31 01:03:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:03:27.463956 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520bdf07-3040-4fa7-b498-66c3df39c6da None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:27.475513 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 65/128] 10.222.0.20 () {58 vars in 1439 bytes} [Wed Aug 31 01:03:27 2022] PUT /identity/v3/projects/82238b119c5840de9fc817856cec36fb/users/7772468996d2491aa2e6bc53000efa74/roles/d009411496b8452b88e77ffb8db015a7 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:03:27.476854 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8dd76e-eb28-4dd8-b2e0-73341613d047 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:27.565764 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:27.597859 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520bdf07-3040-4fa7-b498-66c3df39c6da None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:27.611895 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8dd76e-eb28-4dd8-b2e0-73341613d047 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:27.643458 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bafc13-3600-438a-965e-f96e011c670c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:27.644102 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bafc13-3600-438a-965e-f96e011c670c None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:27.644386 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bafc13-3600-438a-965e-f96e011c670c None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:27.644667 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bafc13-3600-438a-965e-f96e011c670c None admin] PATH_INFO: `/v3/projects/82238b119c5840de9fc817856cec36fb` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:27.645822 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bafc13-3600-438a-965e-f96e011c670c None admin] RBAC: Authorizing `identity:get_project(project_id=82238b119c5840de9fc817856cec36fb)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:27.647280 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bafc13-3600-438a-965e-f96e011c670c None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:27.649238 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 64/129] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Aug 31 01:03:27 2022] GET /identity/v3/projects/82238b119c5840de9fc817856cec36fb => generated 298 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:03:27.650424 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be972b21-b1b4-42e0-8447-f19f3f2b1bde None None] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:27.650706 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be972b21-b1b4-42e0-8447-f19f3f2b1bde None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:27.650977 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be972b21-b1b4-42e0-8447-f19f3f2b1bde None None] PATH_INFO: `/` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:27.652321 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 65/130] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:03:27 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:03:27.654037 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb78e31-7988-40a5-bab2-c327ded6e0ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:27.654682 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be972b21-b1b4-42e0-8447-f19f3f2b1bde None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:27.655068 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb78e31-7988-40a5-bab2-c327ded6e0ae None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:27.655550 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb78e31-7988-40a5-bab2-c327ded6e0ae None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:27.656053 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb78e31-7988-40a5-bab2-c327ded6e0ae None admin] PATH_INFO: `/v3/roles` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:27.656992 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb78e31-7988-40a5-bab2-c327ded6e0ae None admin] RBAC: Adding query filter params (name=member) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:03:27.657504 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb78e31-7988-40a5-bab2-c327ded6e0ae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:27.659111 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb78e31-7988-40a5-bab2-c327ded6e0ae None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:27.669738 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 66/131] 10.222.0.20 () {56 vars in 1101 bytes} [Wed Aug 31 01:03:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:03:27.670773 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb78e31-7988-40a5-bab2-c327ded6e0ae None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:27.707574 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:27.786812 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be972b21-b1b4-42e0-8447-f19f3f2b1bde None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:27.795781 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb78e31-7988-40a5-bab2-c327ded6e0ae None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:27.825926 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f8c5ff-7f19-4c20-b4ba-6aadb5337401 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:27.826884 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f8c5ff-7f19-4c20-b4ba-6aadb5337401 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:27.827373 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:27.827512 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f8c5ff-7f19-4c20-b4ba-6aadb5337401 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:27.827990 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f8c5ff-7f19-4c20-b4ba-6aadb5337401 None admin] PATH_INFO: `/v3/users/1aaea1e3d940441eb7f58357392a5940` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:27.829534 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f8c5ff-7f19-4c20-b4ba-6aadb5337401 None admin] RBAC: Authorizing `identity:get_user(user_id=1aaea1e3d940441eb7f58357392a5940)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:27.831356 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f8c5ff-7f19-4c20-b4ba-6aadb5337401 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:27.833295 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 66/132] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Aug 31 01:03:27 2022] GET /identity/v3/users/1aaea1e3d940441eb7f58357392a5940 => generated 275 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:03:27.836151 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106b55fa-a04d-4c0a-9d22-9280bfab4c22 None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:27.836151 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106b55fa-a04d-4c0a-9d22-9280bfab4c22 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:27.836151 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106b55fa-a04d-4c0a-9d22-9280bfab4c22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:27.839662 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290d5a62-4a94-4e84-9de4-53ff5a2a5a87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:27.840381 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290d5a62-4a94-4e84-9de4-53ff5a2a5a87 None admin] REQUEST_METHOD: `PUT` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:27.840635 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290d5a62-4a94-4e84-9de4-53ff5a2a5a87 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:27.840930 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290d5a62-4a94-4e84-9de4-53ff5a2a5a87 None admin] PATH_INFO: `/v3/projects/82238b119c5840de9fc817856cec36fb/users/1aaea1e3d940441eb7f58357392a5940/roles/bebc77e8dfd54dbba7079f93218e7c7d` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:27.843426 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290d5a62-4a94-4e84-9de4-53ff5a2a5a87 None admin] RBAC: Authorizing `identity:create_grant(project_id=82238b119c5840de9fc817856cec36fb, user_id=1aaea1e3d940441eb7f58357392a5940, role_id=bebc77e8dfd54dbba7079f93218e7c7d)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:27.845590 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290d5a62-4a94-4e84-9de4-53ff5a2a5a87 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:27.856643 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-106b55fa-a04d-4c0a-9d22-9280bfab4c22 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:03:27.857882 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-106b55fa-a04d-4c0a-9d22-9280bfab4c22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:03:27.935381 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 67/133] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:03:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 01:03:27.941763 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106b55fa-a04d-4c0a-9d22-9280bfab4c22 None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:27.956804 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 67/134] 10.222.0.20 () {58 vars in 1439 bytes} [Wed Aug 31 01:03:27 2022] PUT /identity/v3/projects/82238b119c5840de9fc817856cec36fb/users/1aaea1e3d940441eb7f58357392a5940/roles/bebc77e8dfd54dbba7079f93218e7c7d => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:03:27.957761 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290d5a62-4a94-4e84-9de4-53ff5a2a5a87 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:27.997973 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:28.093695 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290d5a62-4a94-4e84-9de4-53ff5a2a5a87 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:28.121268 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106b55fa-a04d-4c0a-9d22-9280bfab4c22 None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:28.135569 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9a935e-5784-481c-87ad-164830f8083a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:28.136223 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9a935e-5784-481c-87ad-164830f8083a None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:28.136556 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9a935e-5784-481c-87ad-164830f8083a None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:28.137424 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9a935e-5784-481c-87ad-164830f8083a None admin] PATH_INFO: `/v3/projects/dc02a04f901440f2a30685de6c0a21d3` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:28.138653 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9a935e-5784-481c-87ad-164830f8083a None admin] RBAC: Authorizing `identity:get_project(project_id=dc02a04f901440f2a30685de6c0a21d3)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:28.140247 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9a935e-5784-481c-87ad-164830f8083a None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:28.142163 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 68/135] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Aug 31 01:03:27 2022] GET /identity/v3/projects/dc02a04f901440f2a30685de6c0a21d3 => generated 312 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:03:28.148362 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9a935e-5784-481c-87ad-164830f8083a None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:28.194643 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81503ed-f9df-4fe6-a1eb-6be17ade89a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:28.195361 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81503ed-f9df-4fe6-a1eb-6be17ade89a2 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:28.195652 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81503ed-f9df-4fe6-a1eb-6be17ade89a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:28.195935 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81503ed-f9df-4fe6-a1eb-6be17ade89a2 None admin] PATH_INFO: `/v3/users/1aaea1e3d940441eb7f58357392a5940` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:28.197934 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81503ed-f9df-4fe6-a1eb-6be17ade89a2 None admin] RBAC: Authorizing `identity:get_user(user_id=1aaea1e3d940441eb7f58357392a5940)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:28.199976 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81503ed-f9df-4fe6-a1eb-6be17ade89a2 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:28.202005 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 68/136] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Aug 31 01:03:27 2022] GET /identity/v3/users/1aaea1e3d940441eb7f58357392a5940 => generated 275 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:03:28.207709 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81503ed-f9df-4fe6-a1eb-6be17ade89a2 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:28.211968 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:28.272405 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9a935e-5784-481c-87ad-164830f8083a None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:28.315680 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3783ae4e-b4a8-486c-9ba7-c9639925859b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:28.316530 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3783ae4e-b4a8-486c-9ba7-c9639925859b None admin] REQUEST_METHOD: `PUT` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:28.317268 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3783ae4e-b4a8-486c-9ba7-c9639925859b None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:28.317268 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3783ae4e-b4a8-486c-9ba7-c9639925859b None admin] PATH_INFO: `/v3/projects/dc02a04f901440f2a30685de6c0a21d3/users/1aaea1e3d940441eb7f58357392a5940/roles/d0b9768f4d014766b9326387f2e11f34` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:28.320105 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3783ae4e-b4a8-486c-9ba7-c9639925859b None admin] RBAC: Authorizing `identity:create_grant(project_id=dc02a04f901440f2a30685de6c0a21d3, user_id=1aaea1e3d940441eb7f58357392a5940, role_id=d0b9768f4d014766b9326387f2e11f34)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:28.322495 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3783ae4e-b4a8-486c-9ba7-c9639925859b None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:28.344087 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81503ed-f9df-4fe6-a1eb-6be17ade89a2 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:28.349841 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:28.384247 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0ff22a-0b5f-47bc-93af-5cea912008b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:28.384942 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0ff22a-0b5f-47bc-93af-5cea912008b6 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:28.385273 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0ff22a-0b5f-47bc-93af-5cea912008b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:28.385601 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0ff22a-0b5f-47bc-93af-5cea912008b6 None admin] PATH_INFO: `/v3/projects/82238b119c5840de9fc817856cec36fb` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:28.386789 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0ff22a-0b5f-47bc-93af-5cea912008b6 None admin] RBAC: Authorizing `identity:get_project(project_id=82238b119c5840de9fc817856cec36fb)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:28.388365 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0ff22a-0b5f-47bc-93af-5cea912008b6 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:28.390442 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 69/137] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Aug 31 01:03:28 2022] GET /identity/v3/projects/82238b119c5840de9fc817856cec36fb => generated 298 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:03:28.391327 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0ff22a-0b5f-47bc-93af-5cea912008b6 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:28.397486 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 69/138] 10.222.0.20 () {58 vars in 1439 bytes} [Wed Aug 31 01:03:28 2022] PUT /identity/v3/projects/dc02a04f901440f2a30685de6c0a21d3/users/1aaea1e3d940441eb7f58357392a5940/roles/d0b9768f4d014766b9326387f2e11f34 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 01:03:28.405155 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3783ae4e-b4a8-486c-9ba7-c9639925859b None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:28.508475 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0ff22a-0b5f-47bc-93af-5cea912008b6 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:28.589553 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51f2609-4104-4b62-afb0-1eb67b99e6e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:28.590215 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51f2609-4104-4b62-afb0-1eb67b99e6e7 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:28.590500 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51f2609-4104-4b62-afb0-1eb67b99e6e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:28.590777 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51f2609-4104-4b62-afb0-1eb67b99e6e7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:28.592894 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51f2609-4104-4b62-afb0-1eb67b99e6e7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:28.594409 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51f2609-4104-4b62-afb0-1eb67b99e6e7 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:28.604435 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 70/139] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 01:03:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:03:28.605927 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc83f248-583c-493a-b72d-e8df08e158ab None None] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:28.606220 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc83f248-583c-493a-b72d-e8df08e158ab None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:28.606501 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc83f248-583c-493a-b72d-e8df08e158ab None None] PATH_INFO: `/` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:28.609578 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 71/140] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:03:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:03:28.610295 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc83f248-583c-493a-b72d-e8df08e158ab None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:28.629098 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3783ae4e-b4a8-486c-9ba7-c9639925859b None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:28.653796 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:28.673538 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:28.714763 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbdf0fc-b01b-4694-afba-149a7d9621d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:28.715448 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbdf0fc-b01b-4694-afba-149a7d9621d7 None admin] REQUEST_METHOD: `PUT` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:28.715735 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbdf0fc-b01b-4694-afba-149a7d9621d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:28.716074 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbdf0fc-b01b-4694-afba-149a7d9621d7 None admin] PATH_INFO: `/v3/projects/82238b119c5840de9fc817856cec36fb/users/1aaea1e3d940441eb7f58357392a5940/roles/d0b9768f4d014766b9326387f2e11f34` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:28.724146 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbdf0fc-b01b-4694-afba-149a7d9621d7 None admin] RBAC: Authorizing `identity:create_grant(project_id=82238b119c5840de9fc817856cec36fb, user_id=1aaea1e3d940441eb7f58357392a5940, role_id=d0b9768f4d014766b9326387f2e11f34)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:28.725644 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbdf0fc-b01b-4694-afba-149a7d9621d7 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:28.727563 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc83f248-583c-493a-b72d-e8df08e158ab None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:28.763672 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00541d3-015b-4e1d-8d95-cf19189c2040 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:28.764290 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00541d3-015b-4e1d-8d95-cf19189c2040 None admin] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:28.764576 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00541d3-015b-4e1d-8d95-cf19189c2040 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:28.764888 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00541d3-015b-4e1d-8d95-cf19189c2040 None admin] PATH_INFO: `/v3/users` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:28.765561 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00541d3-015b-4e1d-8d95-cf19189c2040 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:28.767002 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00541d3-015b-4e1d-8d95-cf19189c2040 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:28.813474 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 70/141] 10.222.0.20 () {58 vars in 1439 bytes} [Wed Aug 31 01:03:28 2022] PUT /identity/v3/projects/82238b119c5840de9fc817856cec36fb/users/1aaea1e3d940441eb7f58357392a5940/roles/d0b9768f4d014766b9326387f2e11f34 => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:03:28.815047 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eccc7d0-b49f-4c96-84bb-ba61adb921e9 None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:28.815351 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eccc7d0-b49f-4c96-84bb-ba61adb921e9 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:28.815637 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eccc7d0-b49f-4c96-84bb-ba61adb921e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:28.847178 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-4eccc7d0-b49f-4c96-84bb-ba61adb921e9 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:03:28.847178 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-4eccc7d0-b49f-4c96-84bb-ba61adb921e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:03:28.871539 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 72/142] 10.222.0.20 () {60 vars in 1132 bytes} [Wed Aug 31 01:03:28 2022] POST /identity/v3/users => generated 283 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:03:28.964159 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 71/143] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:03:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 01:03:29.474593 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00541d3-015b-4e1d-8d95-cf19189c2040 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:29.585952 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:29.592819 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00541d3-015b-4e1d-8d95-cf19189c2040 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:29.640879 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6bd473-e0aa-4c19-bc67-4a53728244d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:29.641518 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6bd473-e0aa-4c19-bc67-4a53728244d4 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:29.641798 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6bd473-e0aa-4c19-bc67-4a53728244d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:29.642072 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6bd473-e0aa-4c19-bc67-4a53728244d4 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:29.666231 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6bd473-e0aa-4c19-bc67-4a53728244d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 01:03:29.671135 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application [None req-6f6bd473-e0aa-4c19-bc67-4a53728244d4 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 01:03:29.671135 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:03:29.671135 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:03:29.671135 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:03:29.671135 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:03:29.671135 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:03:29.671135 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:03:29.671135 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:03:29.671135 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:03:29.671135 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:03:29.671135 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:03:29.671135 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:03:29.671135 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 01:03:29.671135 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 01:03:29.671135 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 01:03:29.671135 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 01:03:29.673341 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 01:03:29.673341 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 01:03:29.673341 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 01:03:29.673341 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 01:03:29.673341 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:03:29.673341 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:03:29.673341 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:03:29.673341 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:03:29.673341 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:03:29.673341 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:03:29.673341 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:03:29.673341 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application with Lock( Aug 31 01:03:29.673341 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:03:29.673341 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:03:29.673341 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:03:29.673341 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:03:29.675280 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:03:29.675280 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:03:29.675280 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:03:29.675280 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:03:29.675280 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 01:03:29.675280 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 01:03:29.675280 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 01:03:29.675280 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 01:03:29.675280 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 01:03:29.675280 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 01:03:29.675280 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 01:03:29.675280 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Aug 31 01:03:29.675280 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 73/144] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Aug 31 01:03:29 2022] GET /identity/v3/roles/reader => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:03:29.680746 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eccc7d0-b49f-4c96-84bb-ba61adb921e9 None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:29.807493 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eccc7d0-b49f-4c96-84bb-ba61adb921e9 None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:29.807493 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:29.846233 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05944df0-0bed-4921-94b2-05ca8bfcc050 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:29.846980 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05944df0-0bed-4921-94b2-05ca8bfcc050 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:29.847143 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05944df0-0bed-4921-94b2-05ca8bfcc050 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:29.847417 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05944df0-0bed-4921-94b2-05ca8bfcc050 None admin] PATH_INFO: `/v3/roles` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:29.848099 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05944df0-0bed-4921-94b2-05ca8bfcc050 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:03:29.848406 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05944df0-0bed-4921-94b2-05ca8bfcc050 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:29.849831 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05944df0-0bed-4921-94b2-05ca8bfcc050 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:29.859115 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 72/145] 10.222.0.20 () {56 vars in 1101 bytes} [Wed Aug 31 01:03:29 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:03:29.864876 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6bd473-e0aa-4c19-bc67-4a53728244d4 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:29.985455 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6bd473-e0aa-4c19-bc67-4a53728244d4 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:30.010065 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af08507f-bee2-48ff-bd0e-39e8ff3be1dd None None] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:30.010700 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af08507f-bee2-48ff-bd0e-39e8ff3be1dd None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:30.011086 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af08507f-bee2-48ff-bd0e-39e8ff3be1dd None None] PATH_INFO: `/` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:30.012830 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 73/146] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:03:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:03:30.029649 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0dacce-37e4-4e7f-8b7c-ed47fca39701 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:30.030556 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0dacce-37e4-4e7f-8b7c-ed47fca39701 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:30.030987 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0dacce-37e4-4e7f-8b7c-ed47fca39701 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:30.031407 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0dacce-37e4-4e7f-8b7c-ed47fca39701 None admin] PATH_INFO: `/v3/users/855cb9a7a6794427b143420fd7d74353` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:30.032715 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0dacce-37e4-4e7f-8b7c-ed47fca39701 None admin] RBAC: Authorizing `identity:get_user(user_id=855cb9a7a6794427b143420fd7d74353)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:30.034389 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0dacce-37e4-4e7f-8b7c-ed47fca39701 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:30.036273 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 74/147] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Aug 31 01:03:29 2022] GET /identity/v3/users/855cb9a7a6794427b143420fd7d74353 => generated 289 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:03:30.040765 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182de8b9-49c2-4766-8d27-cbbec7820a35 None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:30.041056 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182de8b9-49c2-4766-8d27-cbbec7820a35 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:30.041333 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182de8b9-49c2-4766-8d27-cbbec7820a35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:30.045485 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0dacce-37e4-4e7f-8b7c-ed47fca39701 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:30.073575 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-182de8b9-49c2-4766-8d27-cbbec7820a35 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:03:30.076392 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-182de8b9-49c2-4766-8d27-cbbec7820a35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:03:30.096048 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:30.207274 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 74/148] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:03:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 01:03:30.233882 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0dacce-37e4-4e7f-8b7c-ed47fca39701 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:30.279155 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20679bd-440a-4935-9b9d-558f5a622397 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:30.279800 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20679bd-440a-4935-9b9d-558f5a622397 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:30.280129 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20679bd-440a-4935-9b9d-558f5a622397 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:30.280417 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20679bd-440a-4935-9b9d-558f5a622397 None admin] PATH_INFO: `/v3/projects/82238b119c5840de9fc817856cec36fb` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:30.285830 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20679bd-440a-4935-9b9d-558f5a622397 None admin] RBAC: Authorizing `identity:get_project(project_id=82238b119c5840de9fc817856cec36fb)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:30.288388 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20679bd-440a-4935-9b9d-558f5a622397 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:30.290314 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 75/149] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Aug 31 01:03:30 2022] GET /identity/v3/projects/82238b119c5840de9fc817856cec36fb => generated 298 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:03:30.296301 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182de8b9-49c2-4766-8d27-cbbec7820a35 None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:30.418124 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:30.424668 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182de8b9-49c2-4766-8d27-cbbec7820a35 None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:30.464699 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3049e052-5718-48ac-8968-ded8fe8ec186 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:30.465516 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3049e052-5718-48ac-8968-ded8fe8ec186 None admin] REQUEST_METHOD: `PUT` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:30.465814 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3049e052-5718-48ac-8968-ded8fe8ec186 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:30.466090 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3049e052-5718-48ac-8968-ded8fe8ec186 None admin] PATH_INFO: `/v3/projects/82238b119c5840de9fc817856cec36fb/users/855cb9a7a6794427b143420fd7d74353/roles/e1aef2df18ab425a8b78190b0fbfe8ba` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:30.468071 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3049e052-5718-48ac-8968-ded8fe8ec186 None admin] RBAC: Authorizing `identity:create_grant(project_id=82238b119c5840de9fc817856cec36fb, user_id=855cb9a7a6794427b143420fd7d74353, role_id=e1aef2df18ab425a8b78190b0fbfe8ba)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:30.469568 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3049e052-5718-48ac-8968-ded8fe8ec186 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:30.545640 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 75/150] 10.222.0.20 () {58 vars in 1439 bytes} [Wed Aug 31 01:03:30 2022] PUT /identity/v3/projects/82238b119c5840de9fc817856cec36fb/users/855cb9a7a6794427b143420fd7d74353/roles/e1aef2df18ab425a8b78190b0fbfe8ba => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:03:30.649672 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9d8186-0af9-48d2-9d32-6b6e06738f32 None None] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:30.650791 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9d8186-0af9-48d2-9d32-6b6e06738f32 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:30.650791 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9d8186-0af9-48d2-9d32-6b6e06738f32 None None] PATH_INFO: `/` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:30.652439 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 76/151] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:03:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:03:30.666603 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4714b9-02c2-415a-9931-8ee28fac858c None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:30.666919 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4714b9-02c2-415a-9931-8ee28fac858c None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:30.667169 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4714b9-02c2-415a-9931-8ee28fac858c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:30.690205 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-de4714b9-02c2-415a-9931-8ee28fac858c None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:03:30.691335 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-de4714b9-02c2-415a-9931-8ee28fac858c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:03:30.770816 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 76/152] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:03:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 01:03:30.870123 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9d8186-0af9-48d2-9d32-6b6e06738f32 None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:31.009206 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:31.013522 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9d8186-0af9-48d2-9d32-6b6e06738f32 None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:31.054108 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c731742-9636-40ed-ab0f-a0e089f654c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:31.054750 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c731742-9636-40ed-ab0f-a0e089f654c6 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:31.055033 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c731742-9636-40ed-ab0f-a0e089f654c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:31.055306 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c731742-9636-40ed-ab0f-a0e089f654c6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:31.066747 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c731742-9636-40ed-ab0f-a0e089f654c6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 01:03:31.071671 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application [None req-9c731742-9636-40ed-ab0f-a0e089f654c6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 01:03:31.071671 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:03:31.071671 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:03:31.071671 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:03:31.071671 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:03:31.071671 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:03:31.071671 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:03:31.071671 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:03:31.071671 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:03:31.071671 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:03:31.071671 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:03:31.071671 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:03:31.071671 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 01:03:31.071671 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 01:03:31.071671 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 01:03:31.071671 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 01:03:31.072590 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 01:03:31.072590 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 01:03:31.072590 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 01:03:31.072590 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 01:03:31.072590 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:03:31.072590 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:03:31.072590 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:03:31.072590 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:03:31.072590 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:03:31.072590 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:03:31.072590 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:03:31.072590 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application with Lock( Aug 31 01:03:31.072590 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:03:31.072590 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:03:31.072590 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:03:31.072590 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:03:31.074052 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:03:31.074052 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:03:31.074052 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:03:31.074052 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:03:31.074052 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 01:03:31.074052 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 01:03:31.074052 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 01:03:31.074052 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 01:03:31.074052 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 01:03:31.074052 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 01:03:31.074052 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 01:03:31.074052 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Aug 31 01:03:31.074052 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 77/153] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Aug 31 01:03:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:03:31.079142 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4714b9-02c2-415a-9931-8ee28fac858c None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:31.172843 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:31.192674 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4714b9-02c2-415a-9931-8ee28fac858c None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:31.229402 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042b8587-9683-4899-940b-0c232ac1cbfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:31.230022 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042b8587-9683-4899-940b-0c232ac1cbfb None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:31.230305 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042b8587-9683-4899-940b-0c232ac1cbfb None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:31.230580 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042b8587-9683-4899-940b-0c232ac1cbfb None admin] PATH_INFO: `/v3/roles` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:31.231254 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042b8587-9683-4899-940b-0c232ac1cbfb None admin] RBAC: Adding query filter params (name=admin) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:03:31.231558 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042b8587-9683-4899-940b-0c232ac1cbfb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:31.233037 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042b8587-9683-4899-940b-0c232ac1cbfb None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:31.243749 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 77/154] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Aug 31 01:03:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:03:31.247470 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c731742-9636-40ed-ab0f-a0e089f654c6 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:31.376599 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c731742-9636-40ed-ab0f-a0e089f654c6 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:31.413885 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94bd349-3c74-40ba-b1e1-dc689d436377 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:31.414531 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94bd349-3c74-40ba-b1e1-dc689d436377 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:31.414817 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94bd349-3c74-40ba-b1e1-dc689d436377 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:31.415100 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94bd349-3c74-40ba-b1e1-dc689d436377 None admin] PATH_INFO: `/v3/users/7772468996d2491aa2e6bc53000efa74` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:31.420493 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94bd349-3c74-40ba-b1e1-dc689d436377 None admin] RBAC: Authorizing `identity:get_user(user_id=7772468996d2491aa2e6bc53000efa74)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:31.422015 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94bd349-3c74-40ba-b1e1-dc689d436377 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:31.424099 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 78/155] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Aug 31 01:03:31 2022] GET /identity/v3/users/7772468996d2491aa2e6bc53000efa74 => generated 247 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:03:31.429440 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042b8587-9683-4899-940b-0c232ac1cbfb None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:31.458508 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94bd349-3c74-40ba-b1e1-dc689d436377 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:31.496000 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:31.586863 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042b8587-9683-4899-940b-0c232ac1cbfb None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:31.648329 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131ca06c-4aaa-46f7-83e8-c9536f00cd38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:31.649378 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131ca06c-4aaa-46f7-83e8-c9536f00cd38 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:31.649902 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131ca06c-4aaa-46f7-83e8-c9536f00cd38 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:31.650439 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131ca06c-4aaa-46f7-83e8-c9536f00cd38 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:31.654413 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94bd349-3c74-40ba-b1e1-dc689d436377 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:31.685515 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:31.753469 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1017fcb9-ee52-4525-9ee4-7d4d069f0d60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:31.754523 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1017fcb9-ee52-4525-9ee4-7d4d069f0d60 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:31.755058 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1017fcb9-ee52-4525-9ee4-7d4d069f0d60 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:31.755538 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1017fcb9-ee52-4525-9ee4-7d4d069f0d60 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:31.766676 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131ca06c-4aaa-46f7-83e8-c9536f00cd38 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:31.768523 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131ca06c-4aaa-46f7-83e8-c9536f00cd38 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:31.772196 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1017fcb9-ee52-4525-9ee4-7d4d069f0d60 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 01:03:31.778065 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application [None req-1017fcb9-ee52-4525-9ee4-7d4d069f0d60 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 01:03:31.778065 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:03:31.778065 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:03:31.778065 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:03:31.778065 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:03:31.778065 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:03:31.778065 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:03:31.778065 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:03:31.778065 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:03:31.778065 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:03:31.778065 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:03:31.778065 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:03:31.778065 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 01:03:31.778065 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 01:03:31.778065 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 01:03:31.778065 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 01:03:31.778962 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 01:03:31.778962 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 01:03:31.778962 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 01:03:31.778962 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 01:03:31.778962 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:03:31.778962 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:03:31.778962 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:03:31.778962 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:03:31.778962 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:03:31.778962 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:03:31.778962 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:03:31.778962 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application with Lock( Aug 31 01:03:31.778962 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:03:31.778962 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:03:31.778962 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:03:31.778962 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:03:31.779730 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:03:31.779730 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:03:31.779730 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:03:31.779730 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:03:31.779730 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 01:03:31.779730 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 01:03:31.779730 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 01:03:31.779730 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 01:03:31.779730 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 01:03:31.779730 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 01:03:31.779730 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 01:03:31.779730 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Aug 31 01:03:31.786968 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 79/156] 10.222.0.20 () {56 vars in 1114 bytes} [Wed Aug 31 01:03:31 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 329 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:03:31.801735 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1017fcb9-ee52-4525-9ee4-7d4d069f0d60 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:31.919105 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 78/157] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Aug 31 01:03:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 490 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 01:03:31.924480 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131ca06c-4aaa-46f7-83e8-c9536f00cd38 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:31.975003 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:31.993681 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:32.000623 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1017fcb9-ee52-4525-9ee4-7d4d069f0d60 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:32.054803 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e873a83-fdc4-41dd-bf5e-3c5f8fe4125e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:32.055658 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e873a83-fdc4-41dd-bf5e-3c5f8fe4125e None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:32.056146 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e873a83-fdc4-41dd-bf5e-3c5f8fe4125e None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:32.056558 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e873a83-fdc4-41dd-bf5e-3c5f8fe4125e None admin] PATH_INFO: `/v3/roles` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:32.057388 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e873a83-fdc4-41dd-bf5e-3c5f8fe4125e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:03:32.057844 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e873a83-fdc4-41dd-bf5e-3c5f8fe4125e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:32.059374 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e873a83-fdc4-41dd-bf5e-3c5f8fe4125e None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:32.072489 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 80/158] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Aug 31 01:03:31 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:03:32.074223 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c940be4b-7735-4048-b1ec-ab9c89e21a40 None None] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:32.074667 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c940be4b-7735-4048-b1ec-ab9c89e21a40 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:32.075086 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c940be4b-7735-4048-b1ec-ab9c89e21a40 None None] PATH_INFO: `/` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:32.076429 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 81/159] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:03:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:03:32.081565 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c940be4b-7735-4048-b1ec-ab9c89e21a40 None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:32.094854 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131ca06c-4aaa-46f7-83e8-c9536f00cd38 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:32.180448 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f223a6-88b0-49df-90a0-a9caf0ea1962 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:32.181411 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f223a6-88b0-49df-90a0-a9caf0ea1962 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:32.181855 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f223a6-88b0-49df-90a0-a9caf0ea1962 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:32.182273 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f223a6-88b0-49df-90a0-a9caf0ea1962 None admin] PATH_INFO: `/v3/projects/82238b119c5840de9fc817856cec36fb` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:32.183571 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f223a6-88b0-49df-90a0-a9caf0ea1962 None admin] RBAC: Authorizing `identity:get_project(project_id=82238b119c5840de9fc817856cec36fb)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:32.185260 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f223a6-88b0-49df-90a0-a9caf0ea1962 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:32.193496 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 79/160] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Aug 31 01:03:31 2022] GET /identity/v3/projects/82238b119c5840de9fc817856cec36fb => generated 298 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:03:32.194970 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af05577-bd9d-40c5-b79b-21e0d93cef44 None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:32.195413 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af05577-bd9d-40c5-b79b-21e0d93cef44 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:32.195830 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af05577-bd9d-40c5-b79b-21e0d93cef44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:32.226728 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-9af05577-bd9d-40c5-b79b-21e0d93cef44 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:03:32.227287 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c940be4b-7735-4048-b1ec-ab9c89e21a40 None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:32.228402 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-9af05577-bd9d-40c5-b79b-21e0d93cef44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:03:32.266214 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7982e7c3-a7a6-4a55-91e7-ff8c61b9f812 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:32.266845 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7982e7c3-a7a6-4a55-91e7-ff8c61b9f812 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:32.267126 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7982e7c3-a7a6-4a55-91e7-ff8c61b9f812 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:32.267410 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7982e7c3-a7a6-4a55-91e7-ff8c61b9f812 None admin] PATH_INFO: `/v3/users/1aaea1e3d940441eb7f58357392a5940` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:32.268517 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7982e7c3-a7a6-4a55-91e7-ff8c61b9f812 None admin] RBAC: Authorizing `identity:get_user(user_id=1aaea1e3d940441eb7f58357392a5940)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:32.274076 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7982e7c3-a7a6-4a55-91e7-ff8c61b9f812 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:32.275905 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 82/161] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Aug 31 01:03:32 2022] GET /identity/v3/users/1aaea1e3d940441eb7f58357392a5940 => generated 275 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:03:32.276484 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7982e7c3-a7a6-4a55-91e7-ff8c61b9f812 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:32.309711 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:32.340015 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 80/162] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:03:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 01:03:32.341233 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af05577-bd9d-40c5-b79b-21e0d93cef44 None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:32.413219 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:32.467665 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7982e7c3-a7a6-4a55-91e7-ff8c61b9f812 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:32.474401 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af05577-bd9d-40c5-b79b-21e0d93cef44 None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:32.528442 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401ae7ff-937b-4ae5-997d-d57e5c2c5987 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:32.529098 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401ae7ff-937b-4ae5-997d-d57e5c2c5987 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:32.529381 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401ae7ff-937b-4ae5-997d-d57e5c2c5987 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:32.529675 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401ae7ff-937b-4ae5-997d-d57e5c2c5987 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:32.541603 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a246c22c-bb26-4aa6-82ec-36e5a43dfda6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:32.542500 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a246c22c-bb26-4aa6-82ec-36e5a43dfda6 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:32.542985 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a246c22c-bb26-4aa6-82ec-36e5a43dfda6 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:32.543473 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a246c22c-bb26-4aa6-82ec-36e5a43dfda6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:32.544426 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a246c22c-bb26-4aa6-82ec-36e5a43dfda6 None admin] RBAC: Adding query filter params (role.id=d009411496b8452b88e77ffb8db015a7, scope.project.id=82238b119c5840de9fc817856cec36fb, user.id=7772468996d2491aa2e6bc53000efa74) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:03:32.544961 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a246c22c-bb26-4aa6-82ec-36e5a43dfda6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:32.546628 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a246c22c-bb26-4aa6-82ec-36e5a43dfda6 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:32.557540 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 83/163] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Aug 31 01:03:32 2022] GET /identity/v3/role_assignments?user.id=7772468996d2491aa2e6bc53000efa74&scope.project.id=82238b119c5840de9fc817856cec36fb&role.id=d009411496b8452b88e77ffb8db015a7 => generated 608 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:03:32.559079 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a97741-6369-4ea7-8dfd-69de6f347d1c None None] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:32.559577 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a97741-6369-4ea7-8dfd-69de6f347d1c None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:32.560099 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a97741-6369-4ea7-8dfd-69de6f347d1c None None] PATH_INFO: `/` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:32.561549 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 84/164] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:03:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:03:32.567560 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab26b206-b584-466f-b6c7-2140ec149892 None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:32.568237 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab26b206-b584-466f-b6c7-2140ec149892 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:32.568778 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab26b206-b584-466f-b6c7-2140ec149892 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:32.597131 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-ab26b206-b584-466f-b6c7-2140ec149892 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:03:32.598758 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-ab26b206-b584-466f-b6c7-2140ec149892 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:03:32.699168 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401ae7ff-937b-4ae5-997d-d57e5c2c5987 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:32.700926 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401ae7ff-937b-4ae5-997d-d57e5c2c5987 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:32.737092 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 85/165] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:03:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 01:03:32.783720 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:32.848935 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 81/166] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Aug 31 01:03:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 508 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 01:03:32.855347 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab26b206-b584-466f-b6c7-2140ec149892 None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:32.866114 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:32.888254 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401ae7ff-937b-4ae5-997d-d57e5c2c5987 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:33.063594 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab26b206-b584-466f-b6c7-2140ec149892 None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:33.086444 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401ae7ff-937b-4ae5-997d-d57e5c2c5987 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:33.121678 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386b04a8-7ed3-4a7a-9a0a-146feacbb349 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:33.123473 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386b04a8-7ed3-4a7a-9a0a-146feacbb349 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:33.123939 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386b04a8-7ed3-4a7a-9a0a-146feacbb349 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:33.127202 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386b04a8-7ed3-4a7a-9a0a-146feacbb349 None admin] PATH_INFO: `/v3/projects/82238b119c5840de9fc817856cec36fb` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:33.128581 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386b04a8-7ed3-4a7a-9a0a-146feacbb349 None admin] RBAC: Authorizing `identity:get_project(project_id=82238b119c5840de9fc817856cec36fb)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:33.130233 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386b04a8-7ed3-4a7a-9a0a-146feacbb349 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:33.133306 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 86/167] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Aug 31 01:03:32 2022] GET /identity/v3/projects/82238b119c5840de9fc817856cec36fb => generated 298 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:03:33.135818 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8cae61-787f-4a56-8c97-f7b6fb98427e None None] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:33.137032 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8cae61-787f-4a56-8c97-f7b6fb98427e None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:33.138210 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8cae61-787f-4a56-8c97-f7b6fb98427e None None] PATH_INFO: `/` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:33.139866 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 87/168] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:03:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:03:33.141916 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8cae61-787f-4a56-8c97-f7b6fb98427e None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:33.159152 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4646778f-82de-4065-9a85-b05f87f66e0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:33.160507 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4646778f-82de-4065-9a85-b05f87f66e0e None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:33.162725 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4646778f-82de-4065-9a85-b05f87f66e0e None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:33.163203 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4646778f-82de-4065-9a85-b05f87f66e0e None admin] PATH_INFO: `/v3/roles/member` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:33.181697 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4646778f-82de-4065-9a85-b05f87f66e0e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 01:03:33.189674 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application [None req-4646778f-82de-4065-9a85-b05f87f66e0e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 01:03:33.189674 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:03:33.189674 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:03:33.189674 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:03:33.189674 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:03:33.189674 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:03:33.189674 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:03:33.189674 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:03:33.189674 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:03:33.189674 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:03:33.189674 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:03:33.189674 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:03:33.189674 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 01:03:33.189674 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 01:03:33.189674 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 01:03:33.189674 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 01:03:33.190740 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 01:03:33.190740 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 01:03:33.190740 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 01:03:33.190740 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 01:03:33.190740 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:03:33.190740 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:03:33.190740 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:03:33.190740 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:03:33.190740 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:03:33.190740 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:03:33.190740 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:03:33.190740 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application with Lock( Aug 31 01:03:33.190740 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:03:33.190740 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:03:33.190740 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:03:33.190740 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:03:33.191508 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:03:33.191508 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:03:33.191508 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:03:33.191508 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:03:33.191508 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 01:03:33.191508 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 01:03:33.191508 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 01:03:33.191508 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 01:03:33.191508 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 01:03:33.191508 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 01:03:33.191508 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 01:03:33.191508 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Aug 31 01:03:33.196196 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 82/169] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Aug 31 01:03:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:03:33.199084 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb7570e-9e23-4ca9-b8e5-6dc2db8cadca None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:33.199527 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb7570e-9e23-4ca9-b8e5-6dc2db8cadca None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:33.199938 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb7570e-9e23-4ca9-b8e5-6dc2db8cadca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:33.222204 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:33.236631 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-8bb7570e-9e23-4ca9-b8e5-6dc2db8cadca None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:03:33.237834 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-8bb7570e-9e23-4ca9-b8e5-6dc2db8cadca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:03:33.282625 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:33.324104 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8cae61-787f-4a56-8c97-f7b6fb98427e None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:33.347845 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 83/170] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:03:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 01:03:33.348573 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb7570e-9e23-4ca9-b8e5-6dc2db8cadca None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:33.366465 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103e3093-6c97-4cb8-9358-9721087f38bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:33.367155 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103e3093-6c97-4cb8-9358-9721087f38bf None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:33.367655 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103e3093-6c97-4cb8-9358-9721087f38bf None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:33.367940 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103e3093-6c97-4cb8-9358-9721087f38bf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:33.369190 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103e3093-6c97-4cb8-9358-9721087f38bf None admin] RBAC: Adding query filter params (role.id=bebc77e8dfd54dbba7079f93218e7c7d, scope.project.id=82238b119c5840de9fc817856cec36fb, user.id=1aaea1e3d940441eb7f58357392a5940) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:03:33.369505 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103e3093-6c97-4cb8-9358-9721087f38bf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:33.371464 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103e3093-6c97-4cb8-9358-9721087f38bf None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:33.378904 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 88/171] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Aug 31 01:03:33 2022] GET /identity/v3/role_assignments?user.id=1aaea1e3d940441eb7f58357392a5940&scope.project.id=82238b119c5840de9fc817856cec36fb&role.id=bebc77e8dfd54dbba7079f93218e7c7d => generated 608 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:03:33.380207 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103e3093-6c97-4cb8-9358-9721087f38bf None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:33.518885 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103e3093-6c97-4cb8-9358-9721087f38bf None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:33.522873 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb7570e-9e23-4ca9-b8e5-6dc2db8cadca None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:33.562192 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4a4d35-a90c-4836-8324-b4f51dfbb103 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:33.562847 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4a4d35-a90c-4836-8324-b4f51dfbb103 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:33.563132 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4a4d35-a90c-4836-8324-b4f51dfbb103 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:33.563410 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4a4d35-a90c-4836-8324-b4f51dfbb103 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:33.566937 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:33.589746 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4a4d35-a90c-4836-8324-b4f51dfbb103 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 01:03:33.589746 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application [None req-9c4a4d35-a90c-4836-8324-b4f51dfbb103 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 01:03:33.589746 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:03:33.589746 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:03:33.589746 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:03:33.589746 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:03:33.589746 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:03:33.589746 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:03:33.589746 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:03:33.589746 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:03:33.589746 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:03:33.589746 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:03:33.589746 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:03:33.589746 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 01:03:33.590601 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 01:03:33.590601 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 01:03:33.590601 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 01:03:33.590601 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 01:03:33.590601 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 01:03:33.590601 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 01:03:33.590601 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 01:03:33.590601 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:03:33.590601 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:03:33.590601 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:03:33.590601 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:03:33.590601 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:03:33.590601 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:03:33.590601 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:03:33.590601 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application with Lock( Aug 31 01:03:33.590601 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:03:33.590601 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:03:33.591455 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:03:33.591455 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:03:33.591455 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:03:33.591455 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:03:33.591455 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:03:33.591455 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:03:33.591455 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 01:03:33.591455 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 01:03:33.591455 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 01:03:33.591455 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 01:03:33.591455 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 01:03:33.591455 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 01:03:33.591455 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 01:03:33.591455 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Aug 31 01:03:33.591455 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 89/172] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Aug 31 01:03:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:03:33.593384 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4a4d35-a90c-4836-8324-b4f51dfbb103 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:33.617468 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30684b32-a110-4099-85f5-b769acea33df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:33.618107 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30684b32-a110-4099-85f5-b769acea33df None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:33.618392 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30684b32-a110-4099-85f5-b769acea33df None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:33.618739 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30684b32-a110-4099-85f5-b769acea33df None admin] PATH_INFO: `/v3/roles` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:33.619437 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30684b32-a110-4099-85f5-b769acea33df None admin] RBAC: Adding query filter params (name=member) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:03:33.619720 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30684b32-a110-4099-85f5-b769acea33df None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:33.621153 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30684b32-a110-4099-85f5-b769acea33df None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:33.622910 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:33.646685 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 84/173] 10.222.0.20 () {56 vars in 1101 bytes} [Wed Aug 31 01:03:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:03:33.653739 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30684b32-a110-4099-85f5-b769acea33df None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:33.717851 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4a4d35-a90c-4836-8324-b4f51dfbb103 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:33.763168 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099770b2-843f-4f7f-8767-b0f5fe0532bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:33.763789 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099770b2-843f-4f7f-8767-b0f5fe0532bb None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:33.764059 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099770b2-843f-4f7f-8767-b0f5fe0532bb None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:33.764329 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099770b2-843f-4f7f-8767-b0f5fe0532bb None admin] PATH_INFO: `/v3/roles` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:33.765031 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099770b2-843f-4f7f-8767-b0f5fe0532bb None admin] RBAC: Adding query filter params (name=member) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:03:33.765346 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099770b2-843f-4f7f-8767-b0f5fe0532bb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:33.767111 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099770b2-843f-4f7f-8767-b0f5fe0532bb None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:33.776541 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 90/174] 10.222.0.20 () {56 vars in 1101 bytes} [Wed Aug 31 01:03:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:03:33.782175 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099770b2-843f-4f7f-8767-b0f5fe0532bb None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:33.851562 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:33.901196 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099770b2-843f-4f7f-8767-b0f5fe0532bb None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:33.915152 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30684b32-a110-4099-85f5-b769acea33df None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:33.941854 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c09284-aead-4876-badb-85ab2edfd30f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:33.942508 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c09284-aead-4876-badb-85ab2edfd30f None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:33.942799 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c09284-aead-4876-badb-85ab2edfd30f None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:33.943084 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c09284-aead-4876-badb-85ab2edfd30f None admin] PATH_INFO: `/v3/users/1aaea1e3d940441eb7f58357392a5940` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:33.944255 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c09284-aead-4876-badb-85ab2edfd30f None admin] RBAC: Authorizing `identity:get_user(user_id=1aaea1e3d940441eb7f58357392a5940)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:33.945891 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c09284-aead-4876-badb-85ab2edfd30f None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:33.953348 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 91/175] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Aug 31 01:03:33 2022] GET /identity/v3/users/1aaea1e3d940441eb7f58357392a5940 => generated 275 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:03:33.954253 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c09284-aead-4876-badb-85ab2edfd30f None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:33.983541 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70416538-0086-4e45-b98e-b2ca5e51bb43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:33.984183 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70416538-0086-4e45-b98e-b2ca5e51bb43 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:33.984464 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70416538-0086-4e45-b98e-b2ca5e51bb43 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:33.984766 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70416538-0086-4e45-b98e-b2ca5e51bb43 None admin] PATH_INFO: `/v3/users/1aaea1e3d940441eb7f58357392a5940` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:33.985928 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70416538-0086-4e45-b98e-b2ca5e51bb43 None admin] RBAC: Authorizing `identity:get_user(user_id=1aaea1e3d940441eb7f58357392a5940)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:33.987370 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70416538-0086-4e45-b98e-b2ca5e51bb43 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:33.988972 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 85/176] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Aug 31 01:03:33 2022] GET /identity/v3/users/1aaea1e3d940441eb7f58357392a5940 => generated 275 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:03:33.995600 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70416538-0086-4e45-b98e-b2ca5e51bb43 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:34.065747 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c09284-aead-4876-badb-85ab2edfd30f None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:34.093736 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:34.101831 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:34.104191 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ce6a40-3703-4fca-9fb5-b014d726f9df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:34.105012 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ce6a40-3703-4fca-9fb5-b014d726f9df None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:34.110439 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ce6a40-3703-4fca-9fb5-b014d726f9df None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:34.110439 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ce6a40-3703-4fca-9fb5-b014d726f9df None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:34.172502 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70416538-0086-4e45-b98e-b2ca5e51bb43 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:34.178262 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ce6a40-3703-4fca-9fb5-b014d726f9df None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:34.180322 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ce6a40-3703-4fca-9fb5-b014d726f9df None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:34.214949 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c22a8a-84ed-48a6-9ec4-7da6b988340e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:34.215844 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c22a8a-84ed-48a6-9ec4-7da6b988340e None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:34.216235 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c22a8a-84ed-48a6-9ec4-7da6b988340e None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:34.216811 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c22a8a-84ed-48a6-9ec4-7da6b988340e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:34.300490 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 92/177] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Aug 31 01:03:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 01:03:34.300490 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ce6a40-3703-4fca-9fb5-b014d726f9df None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:34.316363 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c22a8a-84ed-48a6-9ec4-7da6b988340e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:34.317891 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c22a8a-84ed-48a6-9ec4-7da6b988340e None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:34.338555 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:34.383817 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:34.413162 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ce6a40-3703-4fca-9fb5-b014d726f9df None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:34.423620 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 86/178] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Aug 31 01:03:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 31 01:03:34.428325 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec0624c-4773-45ce-9eaf-af5c6779b7e1 None None] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:34.428325 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec0624c-4773-45ce-9eaf-af5c6779b7e1 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:34.428325 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec0624c-4773-45ce-9eaf-af5c6779b7e1 None None] PATH_INFO: `/` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:34.428325 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 87/179] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:03:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:03:34.428325 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec0624c-4773-45ce-9eaf-af5c6779b7e1 None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:34.454806 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401b51a0-fa3f-49df-9691-758d9e928d1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:34.455437 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401b51a0-fa3f-49df-9691-758d9e928d1f None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:34.455741 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401b51a0-fa3f-49df-9691-758d9e928d1f None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:34.456016 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401b51a0-fa3f-49df-9691-758d9e928d1f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:34.467322 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401b51a0-fa3f-49df-9691-758d9e928d1f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 01:03:34.472548 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application [None req-401b51a0-fa3f-49df-9691-758d9e928d1f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 01:03:34.472548 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:03:34.472548 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:03:34.472548 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:03:34.472548 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:03:34.472548 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:03:34.472548 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:03:34.472548 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:03:34.472548 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:03:34.472548 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:03:34.472548 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:03:34.472548 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:03:34.472548 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 01:03:34.472548 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 01:03:34.472548 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 01:03:34.472548 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 01:03:34.473558 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 01:03:34.473558 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 01:03:34.473558 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 01:03:34.473558 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 01:03:34.473558 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:03:34.473558 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:03:34.473558 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:03:34.473558 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:03:34.473558 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:03:34.473558 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:03:34.473558 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:03:34.473558 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application with Lock( Aug 31 01:03:34.473558 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:03:34.473558 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:03:34.473558 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:03:34.473558 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:03:34.474334 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:03:34.474334 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:03:34.474334 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:03:34.474334 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:03:34.474334 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 01:03:34.474334 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 01:03:34.474334 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 01:03:34.474334 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 01:03:34.474334 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 01:03:34.474334 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 01:03:34.474334 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 01:03:34.474334 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Aug 31 01:03:34.474334 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 93/180] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Aug 31 01:03:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:03:34.475049 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d6bede-09f4-4ebc-8bb8-864ce984f2e4 None None] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:34.475652 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d6bede-09f4-4ebc-8bb8-864ce984f2e4 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:34.475652 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d6bede-09f4-4ebc-8bb8-864ce984f2e4 None None] PATH_INFO: `/` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:34.476993 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 94/181] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:03:34 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:03:34.477536 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d6bede-09f4-4ebc-8bb8-864ce984f2e4 None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:34.536586 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec0624c-4773-45ce-9eaf-af5c6779b7e1 None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:34.564430 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:34.571493 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08488162-0f5e-4cb0-997a-630de2670f2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:34.572141 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08488162-0f5e-4cb0-997a-630de2670f2d None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:34.572426 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08488162-0f5e-4cb0-997a-630de2670f2d None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:34.572701 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08488162-0f5e-4cb0-997a-630de2670f2d None admin] PATH_INFO: `/v3/projects/82238b119c5840de9fc817856cec36fb` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:34.574412 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08488162-0f5e-4cb0-997a-630de2670f2d None admin] RBAC: Authorizing `identity:get_project(project_id=82238b119c5840de9fc817856cec36fb)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:34.575533 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08488162-0f5e-4cb0-997a-630de2670f2d None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:34.577179 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 88/182] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Aug 31 01:03:34 2022] GET /identity/v3/projects/82238b119c5840de9fc817856cec36fb => generated 298 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:03:34.578299 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b13a06-8ffc-4102-af25-8b8f5a1b63ea None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:34.578578 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b13a06-8ffc-4102-af25-8b8f5a1b63ea None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:34.578849 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b13a06-8ffc-4102-af25-8b8f5a1b63ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:34.580363 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d6bede-09f4-4ebc-8bb8-864ce984f2e4 None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:34.597098 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-74b13a06-8ffc-4102-af25-8b8f5a1b63ea None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:03:34.598272 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-74b13a06-8ffc-4102-af25-8b8f5a1b63ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:03:34.615176 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626c0a4f-3804-43ac-9919-7f798ba30cae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:34.615803 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626c0a4f-3804-43ac-9919-7f798ba30cae None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:34.616086 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626c0a4f-3804-43ac-9919-7f798ba30cae None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:34.616356 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626c0a4f-3804-43ac-9919-7f798ba30cae None admin] PATH_INFO: `/v3/projects/dc02a04f901440f2a30685de6c0a21d3` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:34.617485 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626c0a4f-3804-43ac-9919-7f798ba30cae None admin] RBAC: Authorizing `identity:get_project(project_id=dc02a04f901440f2a30685de6c0a21d3)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:34.618953 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626c0a4f-3804-43ac-9919-7f798ba30cae None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:34.620559 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 95/183] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Aug 31 01:03:34 2022] GET /identity/v3/projects/dc02a04f901440f2a30685de6c0a21d3 => generated 312 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:03:34.621684 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c33d0c4-bf74-488b-a846-d4f9ebc7b282 None None] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:34.621959 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c33d0c4-bf74-488b-a846-d4f9ebc7b282 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:34.622228 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c33d0c4-bf74-488b-a846-d4f9ebc7b282 None None] PATH_INFO: `/` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:34.623571 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 96/184] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:03:34 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:03:34.624078 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c33d0c4-bf74-488b-a846-d4f9ebc7b282 None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:34.629982 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:34.666165 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 89/185] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:03:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 01:03:34.667361 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f762e4-48d1-42f3-95d2-ae791deea3c1 None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:34.667645 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f762e4-48d1-42f3-95d2-ae791deea3c1 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:34.667933 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f762e4-48d1-42f3-95d2-ae791deea3c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:34.693683 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-69f762e4-48d1-42f3-95d2-ae791deea3c1 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:03:34.694853 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-69f762e4-48d1-42f3-95d2-ae791deea3c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:03:34.728752 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c33d0c4-bf74-488b-a846-d4f9ebc7b282 None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:34.760559 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 90/186] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:03:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 01:03:34.761243 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f762e4-48d1-42f3-95d2-ae791deea3c1 None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:34.769154 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5806766-51e8-4af4-af8f-2867b1c1df83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:34.769154 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5806766-51e8-4af4-af8f-2867b1c1df83 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:34.769154 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5806766-51e8-4af4-af8f-2867b1c1df83 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:34.769154 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5806766-51e8-4af4-af8f-2867b1c1df83 None admin] PATH_INFO: `/v3/roles` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:34.769821 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5806766-51e8-4af4-af8f-2867b1c1df83 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:03:34.769821 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5806766-51e8-4af4-af8f-2867b1c1df83 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:34.769821 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5806766-51e8-4af4-af8f-2867b1c1df83 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:34.781691 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 97/187] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Aug 31 01:03:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:03:34.782636 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5806766-51e8-4af4-af8f-2867b1c1df83 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:34.803144 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:34.854166 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:34.862689 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f762e4-48d1-42f3-95d2-ae791deea3c1 None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:34.898506 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5806766-51e8-4af4-af8f-2867b1c1df83 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:34.899631 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9401e718-327f-4c64-8a4d-6bd18e647e03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:34.901161 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9401e718-327f-4c64-8a4d-6bd18e647e03 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:34.901542 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9401e718-327f-4c64-8a4d-6bd18e647e03 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:34.901935 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9401e718-327f-4c64-8a4d-6bd18e647e03 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:34.903607 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9401e718-327f-4c64-8a4d-6bd18e647e03 None admin] RBAC: Adding query filter params (role.id=d0b9768f4d014766b9326387f2e11f34, scope.project.id=82238b119c5840de9fc817856cec36fb, user.id=1aaea1e3d940441eb7f58357392a5940) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:03:34.904017 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9401e718-327f-4c64-8a4d-6bd18e647e03 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:34.905610 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9401e718-327f-4c64-8a4d-6bd18e647e03 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:34.913108 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 91/188] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Aug 31 01:03:34 2022] GET /identity/v3/role_assignments?user.id=1aaea1e3d940441eb7f58357392a5940&scope.project.id=82238b119c5840de9fc817856cec36fb&role.id=d0b9768f4d014766b9326387f2e11f34 => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:03:34.914531 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772edc20-272b-47c5-8c66-48e9d8f94eba None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:34.915205 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772edc20-272b-47c5-8c66-48e9d8f94eba None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:34.915611 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772edc20-272b-47c5-8c66-48e9d8f94eba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:34.939863 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee6dc5c-f942-4807-8383-c4213dca02d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:34.940817 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee6dc5c-f942-4807-8383-c4213dca02d8 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:34.941255 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee6dc5c-f942-4807-8383-c4213dca02d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:34.941704 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee6dc5c-f942-4807-8383-c4213dca02d8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:34.941906 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-772edc20-272b-47c5-8c66-48e9d8f94eba None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:03:34.942777 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee6dc5c-f942-4807-8383-c4213dca02d8 None admin] RBAC: Adding query filter params (role.id=d0b9768f4d014766b9326387f2e11f34, scope.project.id=dc02a04f901440f2a30685de6c0a21d3, user.id=1aaea1e3d940441eb7f58357392a5940) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:03:34.943240 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee6dc5c-f942-4807-8383-c4213dca02d8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:34.943349 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-772edc20-272b-47c5-8c66-48e9d8f94eba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:03:34.944992 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee6dc5c-f942-4807-8383-c4213dca02d8 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:34.952707 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 98/189] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Aug 31 01:03:34 2022] GET /identity/v3/role_assignments?user.id=1aaea1e3d940441eb7f58357392a5940&scope.project.id=dc02a04f901440f2a30685de6c0a21d3&role.id=d0b9768f4d014766b9326387f2e11f34 => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:03:34.958468 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee6dc5c-f942-4807-8383-c4213dca02d8 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:35.032568 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 92/190] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:03:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 01:03:35.033657 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772edc20-272b-47c5-8c66-48e9d8f94eba None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:35.100714 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:35.125586 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee6dc5c-f942-4807-8383-c4213dca02d8 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:35.151154 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:35.167182 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e80f779-7caf-4cd4-b7eb-0ad43b752a12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:35.167950 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e80f779-7caf-4cd4-b7eb-0ad43b752a12 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:35.168299 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e80f779-7caf-4cd4-b7eb-0ad43b752a12 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:35.168677 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e80f779-7caf-4cd4-b7eb-0ad43b752a12 None admin] PATH_INFO: `/v3/users/9b7f934ca2db4023b1d3e2aff511e838` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:35.168843 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772edc20-272b-47c5-8c66-48e9d8f94eba None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:35.183669 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e80f779-7caf-4cd4-b7eb-0ad43b752a12 None admin] RBAC: Authorizing `identity:get_user(user_id=9b7f934ca2db4023b1d3e2aff511e838)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:35.185307 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e80f779-7caf-4cd4-b7eb-0ad43b752a12 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:35.187348 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 99/191] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Aug 31 01:03:34 2022] GET /identity/v3/users/9b7f934ca2db4023b1d3e2aff511e838 => generated 283 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:03:35.192972 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e80f779-7caf-4cd4-b7eb-0ad43b752a12 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:35.212227 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4ed6d9-0fec-4f5a-848f-a1d7aae3dbf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:35.214395 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4ed6d9-0fec-4f5a-848f-a1d7aae3dbf8 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:35.214691 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4ed6d9-0fec-4f5a-848f-a1d7aae3dbf8 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:35.214958 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4ed6d9-0fec-4f5a-848f-a1d7aae3dbf8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:35.216583 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4ed6d9-0fec-4f5a-848f-a1d7aae3dbf8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:35.218875 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4ed6d9-0fec-4f5a-848f-a1d7aae3dbf8 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:35.220447 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 93/192] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 01:03:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:03:35.221387 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4ed6d9-0fec-4f5a-848f-a1d7aae3dbf8 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:35.317583 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e80f779-7caf-4cd4-b7eb-0ad43b752a12 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:35.354508 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4ed6d9-0fec-4f5a-848f-a1d7aae3dbf8 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:35.363417 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3bf381-6f3a-4258-b86a-9e76a2a20ad8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:35.364057 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3bf381-6f3a-4258-b86a-9e76a2a20ad8 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:35.364336 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3bf381-6f3a-4258-b86a-9e76a2a20ad8 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:35.364608 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3bf381-6f3a-4258-b86a-9e76a2a20ad8 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:35.372483 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:35.375976 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3bf381-6f3a-4258-b86a-9e76a2a20ad8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 01:03:35.380839 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application [None req-3f3bf381-6f3a-4258-b86a-9e76a2a20ad8 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 01:03:35.380839 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:03:35.380839 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:03:35.380839 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:03:35.380839 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:03:35.380839 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:03:35.380839 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:03:35.380839 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:03:35.380839 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:03:35.380839 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:03:35.380839 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:03:35.380839 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:03:35.380839 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 01:03:35.380839 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 01:03:35.380839 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 01:03:35.380839 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 01:03:35.386117 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 01:03:35.386117 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 01:03:35.386117 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 01:03:35.386117 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 01:03:35.386117 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:03:35.386117 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:03:35.386117 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:03:35.386117 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:03:35.386117 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:03:35.386117 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:03:35.386117 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:03:35.386117 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application with Lock( Aug 31 01:03:35.386117 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:03:35.386117 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:03:35.386117 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:03:35.386117 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:03:35.387424 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:03:35.387424 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:03:35.387424 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:03:35.387424 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:03:35.387424 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 01:03:35.387424 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 01:03:35.387424 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 01:03:35.387424 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 01:03:35.387424 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 01:03:35.387424 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 01:03:35.387424 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 01:03:35.387424 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Aug 31 01:03:35.387424 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 100/193] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Aug 31 01:03:35 2022] GET /identity/v3/roles/reader => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:03:35.387424 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3bf381-6f3a-4258-b86a-9e76a2a20ad8 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:35.393396 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329362c4-c127-49c1-b2cf-5e0a7e99113b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:35.394001 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329362c4-c127-49c1-b2cf-5e0a7e99113b None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:35.394279 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329362c4-c127-49c1-b2cf-5e0a7e99113b None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:35.394553 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329362c4-c127-49c1-b2cf-5e0a7e99113b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:35.418441 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:35.461975 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329362c4-c127-49c1-b2cf-5e0a7e99113b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:35.463388 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329362c4-c127-49c1-b2cf-5e0a7e99113b None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:35.481731 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3bf381-6f3a-4258-b86a-9e76a2a20ad8 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:35.515384 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b4b275-83a2-4a1e-ad42-455e0c9edaad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:35.515998 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b4b275-83a2-4a1e-ad42-455e0c9edaad None admin] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:35.516286 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b4b275-83a2-4a1e-ad42-455e0c9edaad None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:35.516559 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b4b275-83a2-4a1e-ad42-455e0c9edaad None admin] PATH_INFO: `/v3/users` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:35.517244 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b4b275-83a2-4a1e-ad42-455e0c9edaad None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:35.518727 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b4b275-83a2-4a1e-ad42-455e0c9edaad None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:35.564939 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 94/194] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Aug 31 01:03:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 01:03:35.565781 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329362c4-c127-49c1-b2cf-5e0a7e99113b None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:35.617840 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:35.624739 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 101/195] 10.222.0.20 () {60 vars in 1132 bytes} [Wed Aug 31 01:03:35 2022] POST /identity/v3/users => generated 297 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:03:35.625568 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b4b275-83a2-4a1e-ad42-455e0c9edaad None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:35.668108 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329362c4-c127-49c1-b2cf-5e0a7e99113b None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:35.706387 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:35.708188 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebfa7ce-acaa-4799-bc13-d368988b850e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:35.708863 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebfa7ce-acaa-4799-bc13-d368988b850e None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:35.709134 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebfa7ce-acaa-4799-bc13-d368988b850e None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:35.709403 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebfa7ce-acaa-4799-bc13-d368988b850e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:35.721073 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebfa7ce-acaa-4799-bc13-d368988b850e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 01:03:35.726619 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application [None req-cebfa7ce-acaa-4799-bc13-d368988b850e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 01:03:35.726619 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:03:35.726619 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:03:35.726619 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:03:35.726619 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:03:35.726619 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:03:35.726619 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:03:35.726619 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:03:35.726619 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:03:35.726619 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:03:35.726619 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:03:35.726619 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:03:35.726619 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 01:03:35.726619 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 01:03:35.726619 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 01:03:35.726619 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 01:03:35.727741 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 01:03:35.727741 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 01:03:35.727741 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 01:03:35.727741 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 01:03:35.727741 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:03:35.727741 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:03:35.727741 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:03:35.727741 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:03:35.727741 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:03:35.727741 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:03:35.727741 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:03:35.727741 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application with Lock( Aug 31 01:03:35.727741 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:03:35.727741 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:03:35.727741 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:03:35.727741 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:03:35.729246 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:03:35.729246 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:03:35.729246 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:03:35.729246 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:03:35.729246 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 01:03:35.729246 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 01:03:35.729246 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 01:03:35.729246 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 01:03:35.729246 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 01:03:35.729246 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 01:03:35.729246 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 01:03:35.729246 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Aug 31 01:03:35.729927 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 95/196] 10.222.0.20 () {56 vars in 1114 bytes} [Wed Aug 31 01:03:35 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:03:35.731574 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebfa7ce-acaa-4799-bc13-d368988b850e None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:35.758973 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b4b275-83a2-4a1e-ad42-455e0c9edaad None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:35.794840 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b676c86-e1e9-4a81-b229-474ee8a3df03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:35.795452 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b676c86-e1e9-4a81-b229-474ee8a3df03 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:35.795723 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b676c86-e1e9-4a81-b229-474ee8a3df03 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:35.795996 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b676c86-e1e9-4a81-b229-474ee8a3df03 None admin] PATH_INFO: `/v3/roles` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:35.796663 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b676c86-e1e9-4a81-b229-474ee8a3df03 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:03:35.796992 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b676c86-e1e9-4a81-b229-474ee8a3df03 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:35.798417 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b676c86-e1e9-4a81-b229-474ee8a3df03 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:35.808697 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 102/197] 10.222.0.20 () {56 vars in 1101 bytes} [Wed Aug 31 01:03:35 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:03:35.809656 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b676c86-e1e9-4a81-b229-474ee8a3df03 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:35.856161 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebfa7ce-acaa-4799-bc13-d368988b850e None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:35.874565 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:35.904092 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a769cb14-19cb-412e-82d8-3272b472b78e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:35.904935 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a769cb14-19cb-412e-82d8-3272b472b78e None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:35.905282 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a769cb14-19cb-412e-82d8-3272b472b78e None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:35.905622 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a769cb14-19cb-412e-82d8-3272b472b78e None admin] PATH_INFO: `/v3/projects/b4ae9d03e35549cfbadf1d1001837bb6` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:35.907661 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a769cb14-19cb-412e-82d8-3272b472b78e None admin] RBAC: Authorizing `identity:get_project(project_id=b4ae9d03e35549cfbadf1d1001837bb6)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:35.909318 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a769cb14-19cb-412e-82d8-3272b472b78e None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:35.912191 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 96/198] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Aug 31 01:03:35 2022] GET /identity/v3/projects/b4ae9d03e35549cfbadf1d1001837bb6 => generated 302 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:03:35.913254 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a769cb14-19cb-412e-82d8-3272b472b78e None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:35.941465 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b676c86-e1e9-4a81-b229-474ee8a3df03 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:35.963280 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:35.988677 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adde29a6-f014-483c-9472-7390f715a529 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:35.989347 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adde29a6-f014-483c-9472-7390f715a529 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:35.989630 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adde29a6-f014-483c-9472-7390f715a529 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:35.989804 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adde29a6-f014-483c-9472-7390f715a529 None admin] PATH_INFO: `/v3/roles` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:35.990471 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adde29a6-f014-483c-9472-7390f715a529 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:03:35.990772 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adde29a6-f014-483c-9472-7390f715a529 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:35.992154 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adde29a6-f014-483c-9472-7390f715a529 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:36.002600 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 103/199] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Aug 31 01:03:35 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:03:36.003631 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adde29a6-f014-483c-9472-7390f715a529 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:36.041429 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a769cb14-19cb-412e-82d8-3272b472b78e None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:36.078395 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a80910-cf5c-43fa-b340-10815a817235 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:36.079430 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a80910-cf5c-43fa-b340-10815a817235 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:36.079860 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a80910-cf5c-43fa-b340-10815a817235 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:36.080307 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a80910-cf5c-43fa-b340-10815a817235 None admin] PATH_INFO: `/v3/users/855cb9a7a6794427b143420fd7d74353` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:36.081653 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a80910-cf5c-43fa-b340-10815a817235 None admin] RBAC: Authorizing `identity:get_user(user_id=855cb9a7a6794427b143420fd7d74353)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:36.083287 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a80910-cf5c-43fa-b340-10815a817235 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:36.085062 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 97/200] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Aug 31 01:03:35 2022] GET /identity/v3/users/855cb9a7a6794427b143420fd7d74353 => generated 289 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:03:36.086158 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a80910-cf5c-43fa-b340-10815a817235 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:36.137262 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adde29a6-f014-483c-9472-7390f715a529 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:36.145118 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:36.180570 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1717c89e-a982-4a53-930e-511164ab278a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:36.181274 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1717c89e-a982-4a53-930e-511164ab278a None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:36.181559 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1717c89e-a982-4a53-930e-511164ab278a None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:36.181836 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1717c89e-a982-4a53-930e-511164ab278a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:36.182563 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1717c89e-a982-4a53-930e-511164ab278a None admin] RBAC: Adding query filter params (role.id=d009411496b8452b88e77ffb8db015a7, scope.project.id=b4ae9d03e35549cfbadf1d1001837bb6, user.id=9b7f934ca2db4023b1d3e2aff511e838) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:03:36.182931 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1717c89e-a982-4a53-930e-511164ab278a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:36.184374 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1717c89e-a982-4a53-930e-511164ab278a None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:36.190226 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 104/201] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Aug 31 01:03:36 2022] GET /identity/v3/role_assignments?user.id=9b7f934ca2db4023b1d3e2aff511e838&scope.project.id=b4ae9d03e35549cfbadf1d1001837bb6&role.id=d009411496b8452b88e77ffb8db015a7 => generated 259 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:03:36.191063 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1717c89e-a982-4a53-930e-511164ab278a None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:36.196713 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a80910-cf5c-43fa-b340-10815a817235 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:36.254160 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:36.262982 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edb3485-be09-4a80-a8ed-53ce66a5d4aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:36.263687 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edb3485-be09-4a80-a8ed-53ce66a5d4aa None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:36.263971 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edb3485-be09-4a80-a8ed-53ce66a5d4aa None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:36.264246 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edb3485-be09-4a80-a8ed-53ce66a5d4aa None admin] PATH_INFO: `/v3/users/9b7f934ca2db4023b1d3e2aff511e838` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:36.266034 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edb3485-be09-4a80-a8ed-53ce66a5d4aa None admin] RBAC: Authorizing `identity:get_user(user_id=9b7f934ca2db4023b1d3e2aff511e838)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:36.268653 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edb3485-be09-4a80-a8ed-53ce66a5d4aa None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:36.269307 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 98/202] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Aug 31 01:03:36 2022] GET /identity/v3/users/9b7f934ca2db4023b1d3e2aff511e838 => generated 283 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:03:36.276038 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edb3485-be09-4a80-a8ed-53ce66a5d4aa None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:36.381922 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1717c89e-a982-4a53-930e-511164ab278a None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:36.440175 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0e0ae1-3806-48eb-8643-222103e24d7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:36.440821 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0e0ae1-3806-48eb-8643-222103e24d7c None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:36.441106 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0e0ae1-3806-48eb-8643-222103e24d7c None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:36.441387 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0e0ae1-3806-48eb-8643-222103e24d7c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:36.459002 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edb3485-be09-4a80-a8ed-53ce66a5d4aa None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:36.501813 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:36.536540 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0e0ae1-3806-48eb-8643-222103e24d7c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:36.537955 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0e0ae1-3806-48eb-8643-222103e24d7c None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:36.553445 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cbf891-f747-41ec-8206-931e96db07dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:36.554077 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cbf891-f747-41ec-8206-931e96db07dd None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:36.554360 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cbf891-f747-41ec-8206-931e96db07dd None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:36.554635 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cbf891-f747-41ec-8206-931e96db07dd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:36.586089 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:36.646379 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 105/203] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Aug 31 01:03:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 455 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 01:03:36.652461 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0e0ae1-3806-48eb-8643-222103e24d7c None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:36.718428 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cbf891-f747-41ec-8206-931e96db07dd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:36.719908 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cbf891-f747-41ec-8206-931e96db07dd None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:36.791517 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0e0ae1-3806-48eb-8643-222103e24d7c None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:36.831187 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76029bc-9eda-40b6-b1bd-bc33bec32c9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:36.832616 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76029bc-9eda-40b6-b1bd-bc33bec32c9b None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:36.833313 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76029bc-9eda-40b6-b1bd-bc33bec32c9b None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:36.834990 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76029bc-9eda-40b6-b1bd-bc33bec32c9b None admin] PATH_INFO: `/v3/projects/82238b119c5840de9fc817856cec36fb` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:36.836485 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76029bc-9eda-40b6-b1bd-bc33bec32c9b None admin] RBAC: Authorizing `identity:get_project(project_id=82238b119c5840de9fc817856cec36fb)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:36.839021 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76029bc-9eda-40b6-b1bd-bc33bec32c9b None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:36.842431 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 106/204] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Aug 31 01:03:36 2022] GET /identity/v3/projects/82238b119c5840de9fc817856cec36fb => generated 298 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 01:03:36.848528 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76029bc-9eda-40b6-b1bd-bc33bec32c9b None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:36.869344 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:36.977777 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 99/205] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Aug 31 01:03:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 703 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 01:03:36.982952 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76029bc-9eda-40b6-b1bd-bc33bec32c9b None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:36.983456 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cbf891-f747-41ec-8206-931e96db07dd None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:37.017882 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:37.023198 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60e787c-b553-402c-91bb-2ec3d3af4bb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:37.024078 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60e787c-b553-402c-91bb-2ec3d3af4bb9 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:37.024590 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60e787c-b553-402c-91bb-2ec3d3af4bb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:37.025107 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60e787c-b553-402c-91bb-2ec3d3af4bb9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:37.026351 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60e787c-b553-402c-91bb-2ec3d3af4bb9 None admin] RBAC: Adding query filter params (role.id=e1aef2df18ab425a8b78190b0fbfe8ba, scope.project.id=82238b119c5840de9fc817856cec36fb, user.id=855cb9a7a6794427b143420fd7d74353) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:03:37.026829 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60e787c-b553-402c-91bb-2ec3d3af4bb9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:37.031338 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60e787c-b553-402c-91bb-2ec3d3af4bb9 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:37.035531 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 107/206] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Aug 31 01:03:36 2022] GET /identity/v3/role_assignments?user.id=855cb9a7a6794427b143420fd7d74353&scope.project.id=82238b119c5840de9fc817856cec36fb&role.id=e1aef2df18ab425a8b78190b0fbfe8ba => generated 608 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:03:37.118609 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cbf891-f747-41ec-8206-931e96db07dd None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:37.164085 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42a8506-97c0-4779-b036-f3506707b205 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:37.165037 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42a8506-97c0-4779-b036-f3506707b205 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:37.166141 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42a8506-97c0-4779-b036-f3506707b205 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:37.166565 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42a8506-97c0-4779-b036-f3506707b205 None admin] PATH_INFO: `/v3/projects/b4ae9d03e35549cfbadf1d1001837bb6` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:37.167922 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42a8506-97c0-4779-b036-f3506707b205 None admin] RBAC: Authorizing `identity:get_project(project_id=b4ae9d03e35549cfbadf1d1001837bb6)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:37.169579 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42a8506-97c0-4779-b036-f3506707b205 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:37.171494 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 100/207] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Aug 31 01:03:36 2022] GET /identity/v3/projects/b4ae9d03e35549cfbadf1d1001837bb6 => generated 302 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 01:03:37.177683 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60e787c-b553-402c-91bb-2ec3d3af4bb9 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:37.278254 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:37.307593 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60e787c-b553-402c-91bb-2ec3d3af4bb9 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:37.343435 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c183001f-0385-4e3e-a5e6-075cc06d9b7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:37.344045 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c183001f-0385-4e3e-a5e6-075cc06d9b7e None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:37.344390 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c183001f-0385-4e3e-a5e6-075cc06d9b7e None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:37.344672 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c183001f-0385-4e3e-a5e6-075cc06d9b7e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:37.345507 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c183001f-0385-4e3e-a5e6-075cc06d9b7e None admin] RBAC: Adding query filter params (role.id=bebc77e8dfd54dbba7079f93218e7c7d, scope.project.id=b4ae9d03e35549cfbadf1d1001837bb6, user.id=9b7f934ca2db4023b1d3e2aff511e838) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:03:37.345819 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c183001f-0385-4e3e-a5e6-075cc06d9b7e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:37.347259 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c183001f-0385-4e3e-a5e6-075cc06d9b7e None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:37.353443 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 108/208] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Aug 31 01:03:37 2022] GET /identity/v3/role_assignments?user.id=9b7f934ca2db4023b1d3e2aff511e838&scope.project.id=b4ae9d03e35549cfbadf1d1001837bb6&role.id=bebc77e8dfd54dbba7079f93218e7c7d => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:03:37.812405 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0de631-34f5-40b9-82ab-ef0e5ccb77d5 None None] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:37.813357 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0de631-34f5-40b9-82ab-ef0e5ccb77d5 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:37.813788 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0de631-34f5-40b9-82ab-ef0e5ccb77d5 None None] PATH_INFO: `/` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:37.815793 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 101/209] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:03:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:03:37.820904 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620a3947-0cee-4c47-8854-8329072aa968 None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:37.821196 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620a3947-0cee-4c47-8854-8329072aa968 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:37.821478 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620a3947-0cee-4c47-8854-8329072aa968 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:37.842237 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-620a3947-0cee-4c47-8854-8329072aa968 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:03:37.843524 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-620a3947-0cee-4c47-8854-8329072aa968 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:03:37.956513 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 109/210] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:03:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 01:03:38.015519 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d654c036-db85-464d-b73e-7ca1da920356 None None] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:38.016130 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d654c036-db85-464d-b73e-7ca1da920356 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:38.016600 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d654c036-db85-464d-b73e-7ca1da920356 None None] PATH_INFO: `/` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:38.018487 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 102/211] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:03:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:03:38.023825 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb6a800-b915-46e5-9f2f-cc566290ade7 None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:38.024350 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb6a800-b915-46e5-9f2f-cc566290ade7 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:38.025526 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb6a800-b915-46e5-9f2f-cc566290ade7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:38.043710 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-7fb6a800-b915-46e5-9f2f-cc566290ade7 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:03:38.045234 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-7fb6a800-b915-46e5-9f2f-cc566290ade7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:03:38.113390 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:38.118786 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 110/212] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:03:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 01:03:38.244991 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d654c036-db85-464d-b73e-7ca1da920356 None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:38.340279 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:38.357060 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d654c036-db85-464d-b73e-7ca1da920356 None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:38.361703 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b68a193-51bc-4973-97f2-1b7c47f23dfd None None] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:38.361988 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b68a193-51bc-4973-97f2-1b7c47f23dfd None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:38.362262 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b68a193-51bc-4973-97f2-1b7c47f23dfd None None] PATH_INFO: `/` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:38.363673 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 111/213] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:03:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:03:38.369208 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2c9cef-e8c3-4454-b98b-5057867c954a None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:38.369550 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2c9cef-e8c3-4454-b98b-5057867c954a None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:38.369834 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2c9cef-e8c3-4454-b98b-5057867c954a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:38.390757 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-fa2c9cef-e8c3-4454-b98b-5057867c954a None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:03:38.392691 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-fa2c9cef-e8c3-4454-b98b-5057867c954a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:03:38.401044 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1118b618-025a-482e-99af-298233228c11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:38.401688 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1118b618-025a-482e-99af-298233228c11 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:38.401973 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1118b618-025a-482e-99af-298233228c11 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:38.402251 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1118b618-025a-482e-99af-298233228c11 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:38.413172 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1118b618-025a-482e-99af-298233228c11 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 01:03:38.418153 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application [None req-1118b618-025a-482e-99af-298233228c11 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 01:03:38.418153 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:03:38.418153 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:03:38.418153 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:03:38.418153 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:03:38.418153 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:03:38.418153 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:03:38.418153 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:03:38.418153 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:03:38.418153 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:03:38.418153 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:03:38.418153 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:03:38.418153 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 01:03:38.418153 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 01:03:38.418153 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 01:03:38.418153 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 01:03:38.419015 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 01:03:38.419015 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 01:03:38.419015 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 01:03:38.419015 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 01:03:38.419015 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:03:38.419015 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:03:38.419015 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:03:38.419015 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:03:38.419015 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:03:38.419015 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:03:38.419015 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:03:38.419015 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application with Lock( Aug 31 01:03:38.419015 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:03:38.419015 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:03:38.419015 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:03:38.419015 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:03:38.420473 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:03:38.420473 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:03:38.420473 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:03:38.420473 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:03:38.420473 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 01:03:38.420473 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 01:03:38.420473 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 01:03:38.420473 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 01:03:38.420473 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 01:03:38.420473 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 01:03:38.420473 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 01:03:38.420473 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Aug 31 01:03:38.420473 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 103/214] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Aug 31 01:03:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:03:38.420473 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1118b618-025a-482e-99af-298233228c11 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:38.469382 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 112/215] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:03:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 01:03:38.477916 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2c9cef-e8c3-4454-b98b-5057867c954a None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:38.539734 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1118b618-025a-482e-99af-298233228c11 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:38.587146 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70faf05b-6397-4c42-8715-e108a6fb8c27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:38.588428 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70faf05b-6397-4c42-8715-e108a6fb8c27 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:38.589209 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70faf05b-6397-4c42-8715-e108a6fb8c27 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:38.589692 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70faf05b-6397-4c42-8715-e108a6fb8c27 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:38.591496 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70faf05b-6397-4c42-8715-e108a6fb8c27 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:38.593425 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70faf05b-6397-4c42-8715-e108a6fb8c27 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:38.593730 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2c9cef-e8c3-4454-b98b-5057867c954a None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:38.595598 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 104/216] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 01:03:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:03:38.601944 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70faf05b-6397-4c42-8715-e108a6fb8c27 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:38.619433 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:38.627437 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:38.635834 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0494d1-f502-4445-a4eb-695fa039aea0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:38.636534 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0494d1-f502-4445-a4eb-695fa039aea0 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:38.637183 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0494d1-f502-4445-a4eb-695fa039aea0 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:38.637311 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0494d1-f502-4445-a4eb-695fa039aea0 None admin] PATH_INFO: `/v3/roles` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:38.638075 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0494d1-f502-4445-a4eb-695fa039aea0 None admin] RBAC: Adding query filter params (name=member) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:03:38.638450 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0494d1-f502-4445-a4eb-695fa039aea0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:38.640118 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0494d1-f502-4445-a4eb-695fa039aea0 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:38.649120 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 113/217] 10.222.0.20 () {56 vars in 1101 bytes} [Wed Aug 31 01:03:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:03:38.655882 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0494d1-f502-4445-a4eb-695fa039aea0 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:38.714892 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70faf05b-6397-4c42-8715-e108a6fb8c27 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:38.755043 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163a2ef5-6fcf-4a18-8de1-c327a9e8612e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:38.755901 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163a2ef5-6fcf-4a18-8de1-c327a9e8612e None admin] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:38.756333 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163a2ef5-6fcf-4a18-8de1-c327a9e8612e None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:38.757162 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163a2ef5-6fcf-4a18-8de1-c327a9e8612e None admin] PATH_INFO: `/v3/users` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:38.757994 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163a2ef5-6fcf-4a18-8de1-c327a9e8612e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:38.760169 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163a2ef5-6fcf-4a18-8de1-c327a9e8612e None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:38.780325 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0494d1-f502-4445-a4eb-695fa039aea0 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:38.813955 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf7b225-6a91-4f82-b508-6d80c4dfb7c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:38.814582 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf7b225-6a91-4f82-b508-6d80c4dfb7c7 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:38.814863 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf7b225-6a91-4f82-b508-6d80c4dfb7c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:38.815137 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf7b225-6a91-4f82-b508-6d80c4dfb7c7 None admin] PATH_INFO: `/v3/users/22434d0e26cd419cb313508b0d0ebd7e` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:38.828539 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf7b225-6a91-4f82-b508-6d80c4dfb7c7 None admin] RBAC: Authorizing `identity:get_user(user_id=22434d0e26cd419cb313508b0d0ebd7e)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:38.830165 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf7b225-6a91-4f82-b508-6d80c4dfb7c7 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:38.831834 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 114/218] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Aug 31 01:03:38 2022] GET /identity/v3/users/22434d0e26cd419cb313508b0d0ebd7e => generated 297 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:03:38.832656 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf7b225-6a91-4f82-b508-6d80c4dfb7c7 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:38.844584 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 105/219] 10.222.0.20 () {60 vars in 1132 bytes} [Wed Aug 31 01:03:38 2022] POST /identity/v3/users => generated 297 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 01:03:38.846076 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163a2ef5-6fcf-4a18-8de1-c327a9e8612e None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:38.913690 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:38.954817 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:38.966852 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf7b225-6a91-4f82-b508-6d80c4dfb7c7 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:39.001871 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163a2ef5-6fcf-4a18-8de1-c327a9e8612e None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:39.008026 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e976df85-73dd-473c-88cf-3e6f227e5603 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:39.008849 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e976df85-73dd-473c-88cf-3e6f227e5603 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:39.009260 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e976df85-73dd-473c-88cf-3e6f227e5603 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:39.009631 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e976df85-73dd-473c-88cf-3e6f227e5603 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:39.026705 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e976df85-73dd-473c-88cf-3e6f227e5603 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 01:03:39.032304 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application [None req-e976df85-73dd-473c-88cf-3e6f227e5603 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 01:03:39.032304 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:03:39.032304 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:03:39.032304 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:03:39.032304 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:03:39.032304 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:03:39.032304 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:03:39.032304 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:03:39.032304 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:03:39.032304 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:03:39.032304 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:03:39.032304 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:03:39.032304 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 01:03:39.032304 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 01:03:39.032304 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 01:03:39.032304 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 01:03:39.033232 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 01:03:39.033232 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 01:03:39.033232 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 01:03:39.033232 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 01:03:39.033232 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:03:39.033232 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:03:39.033232 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:03:39.033232 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:03:39.033232 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:03:39.033232 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:03:39.033232 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:03:39.033232 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application with Lock( Aug 31 01:03:39.033232 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:03:39.033232 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:03:39.033232 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:03:39.033232 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:03:39.034223 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:03:39.034223 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:03:39.034223 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:03:39.034223 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:03:39.034223 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 01:03:39.034223 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 01:03:39.034223 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 01:03:39.034223 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 01:03:39.034223 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 01:03:39.034223 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 01:03:39.034223 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 01:03:39.034223 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Aug 31 01:03:39.034223 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 115/220] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Aug 31 01:03:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:03:39.039588 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e976df85-73dd-473c-88cf-3e6f227e5603 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:39.045701 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7507f097-87ec-4885-8c76-2037e5f9d36c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:39.046453 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7507f097-87ec-4885-8c76-2037e5f9d36c None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:39.046858 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7507f097-87ec-4885-8c76-2037e5f9d36c None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:39.047234 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7507f097-87ec-4885-8c76-2037e5f9d36c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:39.125540 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7507f097-87ec-4885-8c76-2037e5f9d36c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:39.127110 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7507f097-87ec-4885-8c76-2037e5f9d36c None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:39.158510 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e976df85-73dd-473c-88cf-3e6f227e5603 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:39.170363 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:39.210564 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9850370a-00d2-4776-b42a-458a3cfb344a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:39.211037 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9850370a-00d2-4776-b42a-458a3cfb344a None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:39.211318 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9850370a-00d2-4776-b42a-458a3cfb344a None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:39.211590 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9850370a-00d2-4776-b42a-458a3cfb344a None admin] PATH_INFO: `/v3/roles` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:39.212882 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9850370a-00d2-4776-b42a-458a3cfb344a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:03:39.213191 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9850370a-00d2-4776-b42a-458a3cfb344a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:39.214596 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9850370a-00d2-4776-b42a-458a3cfb344a None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:39.219319 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:39.227421 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 116/221] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Aug 31 01:03:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 01:03:39.234698 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9850370a-00d2-4776-b42a-458a3cfb344a None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:39.258227 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 106/222] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Aug 31 01:03:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 01:03:39.264782 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7507f097-87ec-4885-8c76-2037e5f9d36c None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:39.372771 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7507f097-87ec-4885-8c76-2037e5f9d36c None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:39.377327 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9850370a-00d2-4776-b42a-458a3cfb344a None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:39.414568 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503cbd95-0a5d-4b57-a057-ecdfff0e0826 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:39.414759 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1720eb40-34b6-4297-978f-721061db3af8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:39.415877 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1720eb40-34b6-4297-978f-721061db3af8 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:39.416166 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1720eb40-34b6-4297-978f-721061db3af8 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:39.416377 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503cbd95-0a5d-4b57-a057-ecdfff0e0826 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:39.416483 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1720eb40-34b6-4297-978f-721061db3af8 None admin] PATH_INFO: `/v3/users/9b7f934ca2db4023b1d3e2aff511e838` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:39.416908 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503cbd95-0a5d-4b57-a057-ecdfff0e0826 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:39.417318 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503cbd95-0a5d-4b57-a057-ecdfff0e0826 None admin] PATH_INFO: `/v3/projects/b4ae9d03e35549cfbadf1d1001837bb6` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:39.417616 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1720eb40-34b6-4297-978f-721061db3af8 None admin] RBAC: Authorizing `identity:get_user(user_id=9b7f934ca2db4023b1d3e2aff511e838)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:39.418749 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503cbd95-0a5d-4b57-a057-ecdfff0e0826 None admin] RBAC: Authorizing `identity:get_project(project_id=b4ae9d03e35549cfbadf1d1001837bb6)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:39.419065 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1720eb40-34b6-4297-978f-721061db3af8 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:39.420537 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503cbd95-0a5d-4b57-a057-ecdfff0e0826 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:39.424760 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 117/223] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Aug 31 01:03:39 2022] GET /identity/v3/users/9b7f934ca2db4023b1d3e2aff511e838 => generated 283 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 01:03:39.424760 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6640c5-fc93-498a-bd14-0005ad98b841 None None] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:39.424760 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6640c5-fc93-498a-bd14-0005ad98b841 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:39.424760 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6640c5-fc93-498a-bd14-0005ad98b841 None None] PATH_INFO: `/` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:39.424760 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 118/224] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:03:39 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:03:39.426859 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 107/225] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Aug 31 01:03:39 2022] GET /identity/v3/projects/b4ae9d03e35549cfbadf1d1001837bb6 => generated 302 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 01:03:39.429705 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6640c5-fc93-498a-bd14-0005ad98b841 None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:39.431716 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b288ec27-16dd-408a-830b-e50af4d188b8 None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:39.432203 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b288ec27-16dd-408a-830b-e50af4d188b8 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:39.432621 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b288ec27-16dd-408a-830b-e50af4d188b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:39.456062 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-b288ec27-16dd-408a-830b-e50af4d188b8 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:03:39.458076 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-b288ec27-16dd-408a-830b-e50af4d188b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:03:39.479156 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:39.502817 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:39.547655 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6640c5-fc93-498a-bd14-0005ad98b841 None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:39.553667 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 108/226] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:03:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 01:03:39.554679 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b288ec27-16dd-408a-830b-e50af4d188b8 None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:39.589119 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6414fc-c2a0-4ef4-8b2b-d817cd11841d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:39.590085 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6414fc-c2a0-4ef4-8b2b-d817cd11841d None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:39.590509 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6414fc-c2a0-4ef4-8b2b-d817cd11841d None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:39.590928 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6414fc-c2a0-4ef4-8b2b-d817cd11841d None admin] PATH_INFO: `/v3/projects/b4ae9d03e35549cfbadf1d1001837bb6` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:39.592257 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6414fc-c2a0-4ef4-8b2b-d817cd11841d None admin] RBAC: Authorizing `identity:get_project(project_id=b4ae9d03e35549cfbadf1d1001837bb6)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:39.594148 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6414fc-c2a0-4ef4-8b2b-d817cd11841d None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:39.595974 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 119/227] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Aug 31 01:03:39 2022] GET /identity/v3/projects/b4ae9d03e35549cfbadf1d1001837bb6 => generated 302 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 01:03:39.601690 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6414fc-c2a0-4ef4-8b2b-d817cd11841d None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:39.666795 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b288ec27-16dd-408a-830b-e50af4d188b8 None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:39.704229 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1714e9ec-edb0-41e5-9174-450013897384 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:39.704890 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1714e9ec-edb0-41e5-9174-450013897384 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:39.705178 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1714e9ec-edb0-41e5-9174-450013897384 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:39.705461 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1714e9ec-edb0-41e5-9174-450013897384 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:39.706205 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6414fc-c2a0-4ef4-8b2b-d817cd11841d None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:39.706318 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1714e9ec-edb0-41e5-9174-450013897384 None admin] RBAC: Adding query filter params (role.id=d0b9768f4d014766b9326387f2e11f34, scope.project.id=b4ae9d03e35549cfbadf1d1001837bb6, user.id=22434d0e26cd419cb313508b0d0ebd7e) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:03:39.706565 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1714e9ec-edb0-41e5-9174-450013897384 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:39.708150 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1714e9ec-edb0-41e5-9174-450013897384 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:39.716174 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:39.718587 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 109/228] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Aug 31 01:03:39 2022] GET /identity/v3/role_assignments?user.id=22434d0e26cd419cb313508b0d0ebd7e&scope.project.id=b4ae9d03e35549cfbadf1d1001837bb6&role.id=d0b9768f4d014766b9326387f2e11f34 => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:03:39.749082 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65abc01-2904-4e0b-8b99-90c1864596ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:39.749788 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65abc01-2904-4e0b-8b99-90c1864596ae None admin] REQUEST_METHOD: `PUT` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:39.750077 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65abc01-2904-4e0b-8b99-90c1864596ae None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:39.750355 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65abc01-2904-4e0b-8b99-90c1864596ae None admin] PATH_INFO: `/v3/projects/b4ae9d03e35549cfbadf1d1001837bb6/users/9b7f934ca2db4023b1d3e2aff511e838/roles/d009411496b8452b88e77ffb8db015a7` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:39.752560 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65abc01-2904-4e0b-8b99-90c1864596ae None admin] RBAC: Authorizing `identity:create_grant(project_id=b4ae9d03e35549cfbadf1d1001837bb6, user_id=9b7f934ca2db4023b1d3e2aff511e838, role_id=d009411496b8452b88e77ffb8db015a7)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:39.754086 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65abc01-2904-4e0b-8b99-90c1864596ae None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:39.831085 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 120/229] 10.222.0.20 () {58 vars in 1439 bytes} [Wed Aug 31 01:03:39 2022] PUT /identity/v3/projects/b4ae9d03e35549cfbadf1d1001837bb6/users/9b7f934ca2db4023b1d3e2aff511e838/roles/d009411496b8452b88e77ffb8db015a7 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:03:40.066551 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1714e9ec-edb0-41e5-9174-450013897384 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:40.108251 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:40.201375 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1714e9ec-edb0-41e5-9174-450013897384 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:40.247243 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac656a3f-b761-412c-88d9-92b5cc7a2ab0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:40.247911 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac656a3f-b761-412c-88d9-92b5cc7a2ab0 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:40.248198 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac656a3f-b761-412c-88d9-92b5cc7a2ab0 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:40.248481 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac656a3f-b761-412c-88d9-92b5cc7a2ab0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:40.266820 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac656a3f-b761-412c-88d9-92b5cc7a2ab0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 01:03:40.272296 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application [None req-ac656a3f-b761-412c-88d9-92b5cc7a2ab0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 01:03:40.272296 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:03:40.272296 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:03:40.272296 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:03:40.272296 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:03:40.272296 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:03:40.272296 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:03:40.272296 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:03:40.272296 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:03:40.272296 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:03:40.272296 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:03:40.272296 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:03:40.272296 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 01:03:40.272296 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 01:03:40.272296 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 01:03:40.272296 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 01:03:40.273214 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 01:03:40.273214 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 01:03:40.273214 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 01:03:40.273214 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 01:03:40.273214 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:03:40.273214 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:03:40.273214 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:03:40.273214 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:03:40.273214 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:03:40.273214 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:03:40.273214 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:03:40.273214 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application with Lock( Aug 31 01:03:40.273214 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:03:40.273214 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:03:40.273214 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:03:40.273214 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:03:40.274030 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:03:40.274030 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:03:40.274030 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:03:40.274030 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:03:40.274030 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 01:03:40.274030 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 01:03:40.274030 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 01:03:40.274030 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 01:03:40.274030 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 01:03:40.274030 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 01:03:40.274030 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 01:03:40.274030 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Aug 31 01:03:40.276852 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 110/230] 10.222.0.20 () {56 vars in 1114 bytes} [Wed Aug 31 01:03:40 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:03:40.281908 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65abc01-2904-4e0b-8b99-90c1864596ae None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:40.406088 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65abc01-2904-4e0b-8b99-90c1864596ae None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:40.449112 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b78b73-1fe7-4441-8ddc-c738c072c80f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:40.449741 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b78b73-1fe7-4441-8ddc-c738c072c80f None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:40.450024 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b78b73-1fe7-4441-8ddc-c738c072c80f None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:40.450355 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b78b73-1fe7-4441-8ddc-c738c072c80f None admin] PATH_INFO: `/v3/roles` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:40.451038 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b78b73-1fe7-4441-8ddc-c738c072c80f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:03:40.451399 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b78b73-1fe7-4441-8ddc-c738c072c80f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:40.452966 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b78b73-1fe7-4441-8ddc-c738c072c80f None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:40.461649 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 121/231] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Aug 31 01:03:40 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:03:40.467891 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac656a3f-b761-412c-88d9-92b5cc7a2ab0 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:40.570383 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:40.585142 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac656a3f-b761-412c-88d9-92b5cc7a2ab0 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:40.636930 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ac0de7-1dd5-4e24-ade2-0f05c26fee11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:40.637636 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ac0de7-1dd5-4e24-ade2-0f05c26fee11 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:40.637925 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ac0de7-1dd5-4e24-ade2-0f05c26fee11 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:40.638205 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ac0de7-1dd5-4e24-ade2-0f05c26fee11 None admin] PATH_INFO: `/v3/users/9b7f934ca2db4023b1d3e2aff511e838` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:40.639380 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ac0de7-1dd5-4e24-ade2-0f05c26fee11 None admin] RBAC: Authorizing `identity:get_user(user_id=9b7f934ca2db4023b1d3e2aff511e838)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:40.640864 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ac0de7-1dd5-4e24-ade2-0f05c26fee11 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:40.642874 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 111/232] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Aug 31 01:03:40 2022] GET /identity/v3/users/9b7f934ca2db4023b1d3e2aff511e838 => generated 283 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:03:40.648069 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b78b73-1fe7-4441-8ddc-c738c072c80f None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:40.692550 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:40.790131 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b78b73-1fe7-4441-8ddc-c738c072c80f None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:40.836220 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4228be2c-9427-4939-ab8b-c68e4a34bacf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:40.836940 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4228be2c-9427-4939-ab8b-c68e4a34bacf None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:40.837252 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4228be2c-9427-4939-ab8b-c68e4a34bacf None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:40.837527 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4228be2c-9427-4939-ab8b-c68e4a34bacf None admin] PATH_INFO: `/v3/projects/b4ae9d03e35549cfbadf1d1001837bb6` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:40.838704 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4228be2c-9427-4939-ab8b-c68e4a34bacf None admin] RBAC: Authorizing `identity:get_project(project_id=b4ae9d03e35549cfbadf1d1001837bb6)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:40.840156 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4228be2c-9427-4939-ab8b-c68e4a34bacf None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:40.842116 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 122/233] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Aug 31 01:03:40 2022] GET /identity/v3/projects/b4ae9d03e35549cfbadf1d1001837bb6 => generated 302 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:03:40.847769 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ac0de7-1dd5-4e24-ade2-0f05c26fee11 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:40.998808 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ac0de7-1dd5-4e24-ade2-0f05c26fee11 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:41.040222 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ed1c06-68e2-4220-9963-dbc81f07e9d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:41.041437 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ed1c06-68e2-4220-9963-dbc81f07e9d6 None admin] REQUEST_METHOD: `PUT` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:41.041720 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ed1c06-68e2-4220-9963-dbc81f07e9d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:41.041997 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ed1c06-68e2-4220-9963-dbc81f07e9d6 None admin] PATH_INFO: `/v3/projects/b4ae9d03e35549cfbadf1d1001837bb6/users/9b7f934ca2db4023b1d3e2aff511e838/roles/bebc77e8dfd54dbba7079f93218e7c7d` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:41.044239 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ed1c06-68e2-4220-9963-dbc81f07e9d6 None admin] RBAC: Authorizing `identity:create_grant(project_id=b4ae9d03e35549cfbadf1d1001837bb6, user_id=9b7f934ca2db4023b1d3e2aff511e838, role_id=bebc77e8dfd54dbba7079f93218e7c7d)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:41.045735 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ed1c06-68e2-4220-9963-dbc81f07e9d6 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:41.155645 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 112/234] 10.222.0.20 () {58 vars in 1439 bytes} [Wed Aug 31 01:03:40 2022] PUT /identity/v3/projects/b4ae9d03e35549cfbadf1d1001837bb6/users/9b7f934ca2db4023b1d3e2aff511e838/roles/bebc77e8dfd54dbba7079f93218e7c7d => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:03:41.267443 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b59ef0-35f0-4f66-aa49-e9b80f5d9515 None None] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:41.267735 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b59ef0-35f0-4f66-aa49-e9b80f5d9515 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:41.267919 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b59ef0-35f0-4f66-aa49-e9b80f5d9515 None None] PATH_INFO: `/` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:41.269299 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 123/235] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:03:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:03:41.275348 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc27e738-260c-4670-826f-365dcab33edb None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:41.275635 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc27e738-260c-4670-826f-365dcab33edb None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:41.275918 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc27e738-260c-4670-826f-365dcab33edb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:41.295928 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-bc27e738-260c-4670-826f-365dcab33edb None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:03:41.297234 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-bc27e738-260c-4670-826f-365dcab33edb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:03:41.312895 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:41.370604 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 113/236] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:03:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 01:03:41.505093 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9706af9d-347c-4c1b-8c3e-8614562ebd59 None None] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:41.505775 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9706af9d-347c-4c1b-8c3e-8614562ebd59 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:41.505775 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9706af9d-347c-4c1b-8c3e-8614562ebd59 None None] PATH_INFO: `/` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:41.506983 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 124/237] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:03:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:03:41.512433 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44567a2a-cc1b-469a-b1ab-1e7e2b75d7ed None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:41.512751 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44567a2a-cc1b-469a-b1ab-1e7e2b75d7ed None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:41.513028 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44567a2a-cc1b-469a-b1ab-1e7e2b75d7ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:41.532075 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-44567a2a-cc1b-469a-b1ab-1e7e2b75d7ed None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:03:41.533234 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-44567a2a-cc1b-469a-b1ab-1e7e2b75d7ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:03:41.608845 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 114/238] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:03:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 01:03:41.679502 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9706af9d-347c-4c1b-8c3e-8614562ebd59 None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:41.816208 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:41.827258 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9706af9d-347c-4c1b-8c3e-8614562ebd59 None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:41.875650 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da0e62c-d0f9-4ad6-b1af-315df3cd2bcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:41.876285 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da0e62c-d0f9-4ad6-b1af-315df3cd2bcb None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:41.876558 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da0e62c-d0f9-4ad6-b1af-315df3cd2bcb None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:41.876864 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da0e62c-d0f9-4ad6-b1af-315df3cd2bcb None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:41.888349 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da0e62c-d0f9-4ad6-b1af-315df3cd2bcb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 01:03:41.894036 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application [None req-7da0e62c-d0f9-4ad6-b1af-315df3cd2bcb None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 01:03:41.894036 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:03:41.894036 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:03:41.894036 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:03:41.894036 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:03:41.894036 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:03:41.894036 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:03:41.894036 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:03:41.894036 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:03:41.894036 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:03:41.894036 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:03:41.894036 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:03:41.894036 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 01:03:41.894036 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 01:03:41.894036 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 01:03:41.894036 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 01:03:41.895407 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 01:03:41.895407 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 01:03:41.895407 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 01:03:41.895407 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 01:03:41.895407 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:03:41.895407 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:03:41.895407 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:03:41.895407 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:03:41.895407 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:03:41.895407 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:03:41.895407 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:03:41.895407 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application with Lock( Aug 31 01:03:41.895407 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:03:41.895407 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:03:41.895407 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:03:41.895407 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:03:41.896549 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:03:41.896549 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:03:41.896549 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:03:41.896549 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:03:41.896549 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 01:03:41.896549 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 01:03:41.896549 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 01:03:41.896549 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 01:03:41.896549 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 01:03:41.896549 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 01:03:41.896549 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 01:03:41.896549 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Aug 31 01:03:41.896549 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 125/239] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Aug 31 01:03:41 2022] GET /identity/v3/roles/reader => generated 84 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:03:41.909901 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44567a2a-cc1b-469a-b1ab-1e7e2b75d7ed None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:42.046409 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2721c064-6c30-498f-830d-eeb9c44feb3c None None] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:42.046754 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2721c064-6c30-498f-830d-eeb9c44feb3c None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:42.046972 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2721c064-6c30-498f-830d-eeb9c44feb3c None None] PATH_INFO: `/` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:42.049015 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 126/240] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:03:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 01:03:42.054020 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdc6d02-b979-4118-9a55-b2c4a893d2df None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:42.054506 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdc6d02-b979-4118-9a55-b2c4a893d2df None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:42.054873 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdc6d02-b979-4118-9a55-b2c4a893d2df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:42.056476 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:42.079579 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44567a2a-cc1b-469a-b1ab-1e7e2b75d7ed None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:42.080502 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-ccdc6d02-b979-4118-9a55-b2c4a893d2df None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:03:42.082899 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-ccdc6d02-b979-4118-9a55-b2c4a893d2df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:03:42.118135 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c9f50c-aa7f-4213-9f31-70e8a29f1db7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:42.119169 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c9f50c-aa7f-4213-9f31-70e8a29f1db7 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:42.119573 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c9f50c-aa7f-4213-9f31-70e8a29f1db7 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:42.119937 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c9f50c-aa7f-4213-9f31-70e8a29f1db7 None admin] PATH_INFO: `/v3/roles` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:42.120763 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c9f50c-aa7f-4213-9f31-70e8a29f1db7 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:03:42.121166 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c9f50c-aa7f-4213-9f31-70e8a29f1db7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:42.122985 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c9f50c-aa7f-4213-9f31-70e8a29f1db7 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:42.136270 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 115/241] 10.222.0.20 () {56 vars in 1101 bytes} [Wed Aug 31 01:03:41 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:03:42.136270 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c9f50c-aa7f-4213-9f31-70e8a29f1db7 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:42.168854 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 127/242] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:03:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 01:03:42.170002 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdc6d02-b979-4118-9a55-b2c4a893d2df None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:42.242384 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c9f50c-aa7f-4213-9f31-70e8a29f1db7 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:42.278436 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef9c7f1-3d17-4ec6-b155-7a602876ccb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:42.279156 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef9c7f1-3d17-4ec6-b155-7a602876ccb2 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:42.279440 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef9c7f1-3d17-4ec6-b155-7a602876ccb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:42.279708 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef9c7f1-3d17-4ec6-b155-7a602876ccb2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:42.280846 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef9c7f1-3d17-4ec6-b155-7a602876ccb2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:42.282420 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef9c7f1-3d17-4ec6-b155-7a602876ccb2 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:42.284455 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 116/243] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 01:03:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:03:42.298329 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef9c7f1-3d17-4ec6-b155-7a602876ccb2 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:42.321271 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:42.333437 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:42.349151 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdc6d02-b979-4118-9a55-b2c4a893d2df None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:42.394753 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dd6a5d-281a-459b-bbfa-d3c85ed9c924 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:42.395395 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dd6a5d-281a-459b-bbfa-d3c85ed9c924 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:42.395679 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dd6a5d-281a-459b-bbfa-d3c85ed9c924 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:42.395955 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dd6a5d-281a-459b-bbfa-d3c85ed9c924 None admin] PATH_INFO: `/v3/users/49f42887f51c497d8ae8ff3f7eb71e07` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:42.420900 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65dd6a5d-281a-459b-bbfa-d3c85ed9c924 None admin] RBAC: Authorizing `identity:get_user(user_id=49f42887f51c497d8ae8ff3f7eb71e07)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:42.422397 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65dd6a5d-281a-459b-bbfa-d3c85ed9c924 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:42.424395 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 128/244] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Aug 31 01:03:42 2022] GET /identity/v3/users/49f42887f51c497d8ae8ff3f7eb71e07 => generated 297 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:03:42.430133 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0c9d4c-9d03-4af7-a04a-05c9d5e59d20 None None] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:42.430694 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0c9d4c-9d03-4af7-a04a-05c9d5e59d20 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:42.431702 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0c9d4c-9d03-4af7-a04a-05c9d5e59d20 None None] PATH_INFO: `/` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:42.433188 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 129/245] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:03:42 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:03:42.434071 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0c9d4c-9d03-4af7-a04a-05c9d5e59d20 None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:42.450386 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef9c7f1-3d17-4ec6-b155-7a602876ccb2 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:42.486460 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814ccf4a-ef9b-4424-a7ee-056fdf2e6386 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:42.487078 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814ccf4a-ef9b-4424-a7ee-056fdf2e6386 None admin] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:42.487500 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814ccf4a-ef9b-4424-a7ee-056fdf2e6386 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:42.487847 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814ccf4a-ef9b-4424-a7ee-056fdf2e6386 None admin] PATH_INFO: `/v3/users` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:42.488563 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814ccf4a-ef9b-4424-a7ee-056fdf2e6386 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:42.490058 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814ccf4a-ef9b-4424-a7ee-056fdf2e6386 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:42.544060 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0c9d4c-9d03-4af7-a04a-05c9d5e59d20 None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:42.566098 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 117/246] 10.222.0.20 () {60 vars in 1132 bytes} [Wed Aug 31 01:03:42 2022] POST /identity/v3/users => generated 293 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:03:42.567938 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e721bc21-e932-4c10-9d8b-6fc2eaff8f6c None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:42.568597 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e721bc21-e932-4c10-9d8b-6fc2eaff8f6c None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:42.568962 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e721bc21-e932-4c10-9d8b-6fc2eaff8f6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:42.583785 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bb4ee9-ed46-4cd2-a0c1-7b815d1113d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:42.584647 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bb4ee9-ed46-4cd2-a0c1-7b815d1113d2 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:42.585094 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bb4ee9-ed46-4cd2-a0c1-7b815d1113d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:42.585508 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bb4ee9-ed46-4cd2-a0c1-7b815d1113d2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:42.594227 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-e721bc21-e932-4c10-9d8b-6fc2eaff8f6c None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:03:42.595732 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-e721bc21-e932-4c10-9d8b-6fc2eaff8f6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:03:42.605937 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:42.673065 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bb4ee9-ed46-4cd2-a0c1-7b815d1113d2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:42.674835 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bb4ee9-ed46-4cd2-a0c1-7b815d1113d2 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:42.689806 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:42.701532 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 118/247] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:03:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 01:03:42.702334 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e721bc21-e932-4c10-9d8b-6fc2eaff8f6c None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:42.794505 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 130/248] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Aug 31 01:03:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 01:03:42.800317 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bb4ee9-ed46-4cd2-a0c1-7b815d1113d2 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:42.839108 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e721bc21-e932-4c10-9d8b-6fc2eaff8f6c None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:42.853752 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:42.887727 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c133ae-ca81-4db2-a28c-cc7df87f8c9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:42.888875 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c133ae-ca81-4db2-a28c-cc7df87f8c9b None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:42.889163 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c133ae-ca81-4db2-a28c-cc7df87f8c9b None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:42.889439 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c133ae-ca81-4db2-a28c-cc7df87f8c9b None admin] PATH_INFO: `/v3/roles/member` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:42.903264 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c133ae-ca81-4db2-a28c-cc7df87f8c9b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 01:03:42.909742 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application [None req-97c133ae-ca81-4db2-a28c-cc7df87f8c9b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 01:03:42.909742 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:03:42.909742 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:03:42.909742 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:03:42.909742 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:03:42.909742 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:03:42.909742 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:03:42.909742 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:03:42.909742 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:03:42.909742 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:03:42.909742 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:03:42.909742 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:03:42.909742 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 01:03:42.909742 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 01:03:42.909742 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 01:03:42.909742 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 01:03:42.910720 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 01:03:42.910720 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 01:03:42.910720 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 01:03:42.910720 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 01:03:42.910720 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:03:42.910720 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:03:42.910720 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:03:42.910720 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:03:42.910720 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:03:42.910720 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:03:42.910720 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:03:42.910720 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application with Lock( Aug 31 01:03:42.910720 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:03:42.910720 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:03:42.910720 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:03:42.910720 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:03:42.911506 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:03:42.911506 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:03:42.911506 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:03:42.911506 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:03:42.911506 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 01:03:42.911506 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 01:03:42.911506 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 01:03:42.911506 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 01:03:42.911506 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 01:03:42.911506 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 01:03:42.911506 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 01:03:42.911506 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Aug 31 01:03:42.913074 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 119/249] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Aug 31 01:03:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:03:42.920033 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c133ae-ca81-4db2-a28c-cc7df87f8c9b None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:42.935430 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bb4ee9-ed46-4cd2-a0c1-7b815d1113d2 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:42.975635 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7030a570-0c84-4ca8-9c18-d15ec140464a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:42.976100 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7030a570-0c84-4ca8-9c18-d15ec140464a None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:42.976377 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7030a570-0c84-4ca8-9c18-d15ec140464a None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:42.976647 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7030a570-0c84-4ca8-9c18-d15ec140464a None admin] PATH_INFO: `/v3/projects/b4ae9d03e35549cfbadf1d1001837bb6` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:42.980399 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7030a570-0c84-4ca8-9c18-d15ec140464a None admin] RBAC: Authorizing `identity:get_project(project_id=b4ae9d03e35549cfbadf1d1001837bb6)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:42.982219 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7030a570-0c84-4ca8-9c18-d15ec140464a None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:42.983988 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 131/250] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Aug 31 01:03:42 2022] GET /identity/v3/projects/b4ae9d03e35549cfbadf1d1001837bb6 => generated 302 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:03:42.989761 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7030a570-0c84-4ca8-9c18-d15ec140464a None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:42.997438 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:43.052837 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c133ae-ca81-4db2-a28c-cc7df87f8c9b None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:43.098916 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd66f4c-362b-4055-8b18-26a0ceb34c97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:43.099462 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd66f4c-362b-4055-8b18-26a0ceb34c97 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:43.099743 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd66f4c-362b-4055-8b18-26a0ceb34c97 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:43.100015 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd66f4c-362b-4055-8b18-26a0ceb34c97 None admin] PATH_INFO: `/v3/roles` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:43.100694 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd66f4c-362b-4055-8b18-26a0ceb34c97 None admin] RBAC: Adding query filter params (name=member) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:03:43.101024 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd66f4c-362b-4055-8b18-26a0ceb34c97 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:43.102501 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd66f4c-362b-4055-8b18-26a0ceb34c97 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:43.113860 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 120/251] 10.222.0.20 () {56 vars in 1101 bytes} [Wed Aug 31 01:03:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 01:03:43.115496 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd66f4c-362b-4055-8b18-26a0ceb34c97 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:43.118021 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7030a570-0c84-4ca8-9c18-d15ec140464a None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:43.138948 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:43.164425 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da85090-47ca-4f2a-b650-75269208ae4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:43.165077 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da85090-47ca-4f2a-b650-75269208ae4e None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:43.165392 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da85090-47ca-4f2a-b650-75269208ae4e None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:43.165666 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da85090-47ca-4f2a-b650-75269208ae4e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:43.166398 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da85090-47ca-4f2a-b650-75269208ae4e None admin] RBAC: Adding query filter params (role.id=e1aef2df18ab425a8b78190b0fbfe8ba, scope.project.id=b4ae9d03e35549cfbadf1d1001837bb6, user.id=49f42887f51c497d8ae8ff3f7eb71e07) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:03:43.166706 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da85090-47ca-4f2a-b650-75269208ae4e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:43.168166 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da85090-47ca-4f2a-b650-75269208ae4e None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:43.174761 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 132/252] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Aug 31 01:03:42 2022] GET /identity/v3/role_assignments?user.id=49f42887f51c497d8ae8ff3f7eb71e07&scope.project.id=b4ae9d03e35549cfbadf1d1001837bb6&role.id=e1aef2df18ab425a8b78190b0fbfe8ba => generated 259 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:03:43.175596 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da85090-47ca-4f2a-b650-75269208ae4e None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:43.249394 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd66f4c-362b-4055-8b18-26a0ceb34c97 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:43.272513 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:43.310283 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c68dcf8-9ff7-4a02-b368-51d05cfb8cda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:43.313487 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c68dcf8-9ff7-4a02-b368-51d05cfb8cda None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:43.314014 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c68dcf8-9ff7-4a02-b368-51d05cfb8cda None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:43.314492 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c68dcf8-9ff7-4a02-b368-51d05cfb8cda None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:43.341916 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da85090-47ca-4f2a-b650-75269208ae4e None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:43.344762 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c68dcf8-9ff7-4a02-b368-51d05cfb8cda None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 01:03:43.357334 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application [None req-9c68dcf8-9ff7-4a02-b368-51d05cfb8cda None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 01:03:43.357334 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:03:43.357334 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:03:43.357334 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:03:43.357334 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:03:43.357334 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:03:43.357334 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:03:43.357334 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:03:43.357334 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:03:43.357334 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:03:43.357334 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:03:43.357334 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:03:43.357334 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 01:03:43.357334 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 01:03:43.357334 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 01:03:43.357334 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 01:03:43.358275 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 01:03:43.358275 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 01:03:43.358275 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 01:03:43.358275 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 01:03:43.358275 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:03:43.358275 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:03:43.358275 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:03:43.358275 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:03:43.358275 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:03:43.358275 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:03:43.358275 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:03:43.358275 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application with Lock( Aug 31 01:03:43.358275 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:03:43.358275 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:03:43.358275 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:03:43.358275 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:03:43.359088 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:03:43.359088 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:03:43.359088 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:03:43.359088 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:03:43.359088 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 01:03:43.359088 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 01:03:43.359088 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 01:03:43.359088 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 01:03:43.359088 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 01:03:43.359088 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 01:03:43.359088 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 01:03:43.359088 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Aug 31 01:03:43.361186 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 121/253] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Aug 31 01:03:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:03:43.368693 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c68dcf8-9ff7-4a02-b368-51d05cfb8cda None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:43.415087 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988faf7a-1958-4c31-8f85-8215b557ac4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:43.416071 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988faf7a-1958-4c31-8f85-8215b557ac4e None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:43.416515 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988faf7a-1958-4c31-8f85-8215b557ac4e None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:43.416970 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988faf7a-1958-4c31-8f85-8215b557ac4e None admin] PATH_INFO: `/v3/users/22434d0e26cd419cb313508b0d0ebd7e` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:43.418667 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988faf7a-1958-4c31-8f85-8215b557ac4e None admin] RBAC: Authorizing `identity:get_user(user_id=22434d0e26cd419cb313508b0d0ebd7e)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:43.420741 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988faf7a-1958-4c31-8f85-8215b557ac4e None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:43.423148 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 133/254] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Aug 31 01:03:43 2022] GET /identity/v3/users/22434d0e26cd419cb313508b0d0ebd7e => generated 297 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:03:43.429545 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988faf7a-1958-4c31-8f85-8215b557ac4e None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:43.518194 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:43.581277 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988faf7a-1958-4c31-8f85-8215b557ac4e None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:43.582990 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c68dcf8-9ff7-4a02-b368-51d05cfb8cda None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:43.621419 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7db7e99-ff4c-4f2b-94cb-a1417aa49ca6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:43.622440 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7db7e99-ff4c-4f2b-94cb-a1417aa49ca6 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:43.622874 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7db7e99-ff4c-4f2b-94cb-a1417aa49ca6 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:43.623284 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7db7e99-ff4c-4f2b-94cb-a1417aa49ca6 None admin] PATH_INFO: `/v3/projects/b4ae9d03e35549cfbadf1d1001837bb6` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:43.624800 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7db7e99-ff4c-4f2b-94cb-a1417aa49ca6 None admin] RBAC: Authorizing `identity:get_project(project_id=b4ae9d03e35549cfbadf1d1001837bb6)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:43.626609 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7db7e99-ff4c-4f2b-94cb-a1417aa49ca6 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:43.629486 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 134/255] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Aug 31 01:03:43 2022] GET /identity/v3/projects/b4ae9d03e35549cfbadf1d1001837bb6 => generated 302 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:03:43.630957 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82469fbe-131f-49d3-a36f-dfebd52a4e80 None None] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:43.631382 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82469fbe-131f-49d3-a36f-dfebd52a4e80 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:43.631798 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82469fbe-131f-49d3-a36f-dfebd52a4e80 None None] PATH_INFO: `/` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:43.633686 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 135/256] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:03:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:03:43.640556 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2fcf8e-5f88-43fb-ba1e-2220dff2b19f None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:43.641039 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2fcf8e-5f88-43fb-ba1e-2220dff2b19f None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:43.641626 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2fcf8e-5f88-43fb-ba1e-2220dff2b19f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:43.653497 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1dda67-4f96-436c-8199-1b932b6b3d7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:43.654139 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1dda67-4f96-436c-8199-1b932b6b3d7a None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:43.654430 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1dda67-4f96-436c-8199-1b932b6b3d7a None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:43.654719 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1dda67-4f96-436c-8199-1b932b6b3d7a None admin] PATH_INFO: `/v3/roles` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:43.655409 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1dda67-4f96-436c-8199-1b932b6b3d7a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:03:43.655723 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1dda67-4f96-436c-8199-1b932b6b3d7a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:43.657180 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1dda67-4f96-436c-8199-1b932b6b3d7a None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:43.662431 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-2e2fcf8e-5f88-43fb-ba1e-2220dff2b19f None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:03:43.663834 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-2e2fcf8e-5f88-43fb-ba1e-2220dff2b19f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:03:43.676518 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 122/257] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Aug 31 01:03:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:03:43.677189 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1dda67-4f96-436c-8199-1b932b6b3d7a None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:43.741596 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 136/258] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:03:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 01:03:43.748467 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2fcf8e-5f88-43fb-ba1e-2220dff2b19f None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:43.765934 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:43.841677 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:43.907492 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1dda67-4f96-436c-8199-1b932b6b3d7a None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:43.999906 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2fcf8e-5f88-43fb-ba1e-2220dff2b19f None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:44.005400 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e5a1b7-29bf-4c64-8225-187e06f8245e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:44.006080 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e5a1b7-29bf-4c64-8225-187e06f8245e None admin] REQUEST_METHOD: `PUT` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:44.006363 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e5a1b7-29bf-4c64-8225-187e06f8245e None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:44.006643 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e5a1b7-29bf-4c64-8225-187e06f8245e None admin] PATH_INFO: `/v3/projects/b4ae9d03e35549cfbadf1d1001837bb6/users/22434d0e26cd419cb313508b0d0ebd7e/roles/d0b9768f4d014766b9326387f2e11f34` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:44.008633 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e5a1b7-29bf-4c64-8225-187e06f8245e None admin] RBAC: Authorizing `identity:create_grant(project_id=b4ae9d03e35549cfbadf1d1001837bb6, user_id=22434d0e26cd419cb313508b0d0ebd7e, role_id=d0b9768f4d014766b9326387f2e11f34)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:44.010164 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e5a1b7-29bf-4c64-8225-187e06f8245e None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:44.090563 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93b1bc0-9061-4e69-9f1a-e08e0324eb52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:44.091210 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93b1bc0-9061-4e69-9f1a-e08e0324eb52 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:44.091495 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93b1bc0-9061-4e69-9f1a-e08e0324eb52 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:44.091774 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93b1bc0-9061-4e69-9f1a-e08e0324eb52 None admin] PATH_INFO: `/v3/users/9b7f934ca2db4023b1d3e2aff511e838` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:44.093064 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93b1bc0-9061-4e69-9f1a-e08e0324eb52 None admin] RBAC: Authorizing `identity:get_user(user_id=9b7f934ca2db4023b1d3e2aff511e838)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:44.094507 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93b1bc0-9061-4e69-9f1a-e08e0324eb52 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:44.096110 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 137/259] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Aug 31 01:03:43 2022] GET /identity/v3/users/9b7f934ca2db4023b1d3e2aff511e838 => generated 283 bytes in 354 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:03:44.097267 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93b1bc0-9061-4e69-9f1a-e08e0324eb52 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:44.103102 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 123/260] 10.222.0.20 () {58 vars in 1439 bytes} [Wed Aug 31 01:03:43 2022] PUT /identity/v3/projects/b4ae9d03e35549cfbadf1d1001837bb6/users/22434d0e26cd419cb313508b0d0ebd7e/roles/d0b9768f4d014766b9326387f2e11f34 => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:03:44.108334 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e5a1b7-29bf-4c64-8225-187e06f8245e None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:44.221143 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93b1bc0-9061-4e69-9f1a-e08e0324eb52 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:44.234244 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:44.262019 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23731a25-cb69-45b8-8414-6cfa1739e0bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:44.262965 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23731a25-cb69-45b8-8414-6cfa1739e0bf None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:44.263256 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23731a25-cb69-45b8-8414-6cfa1739e0bf None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:44.263533 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23731a25-cb69-45b8-8414-6cfa1739e0bf None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:44.277717 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23731a25-cb69-45b8-8414-6cfa1739e0bf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 01:03:44.283376 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application [None req-23731a25-cb69-45b8-8414-6cfa1739e0bf None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 01:03:44.283376 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:03:44.283376 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:03:44.283376 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:03:44.283376 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:03:44.283376 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:03:44.283376 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:03:44.283376 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:03:44.283376 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:03:44.283376 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:03:44.283376 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:03:44.283376 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:03:44.283376 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 01:03:44.283376 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 01:03:44.283376 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 01:03:44.283376 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 01:03:44.284307 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 01:03:44.284307 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 01:03:44.284307 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 01:03:44.284307 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 01:03:44.284307 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:03:44.284307 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:03:44.284307 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:03:44.284307 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:03:44.284307 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:03:44.284307 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:03:44.284307 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:03:44.284307 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application with Lock( Aug 31 01:03:44.284307 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:03:44.284307 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:03:44.284307 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:03:44.284307 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:03:44.285201 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:03:44.285201 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:03:44.285201 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:03:44.285201 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:03:44.285201 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 01:03:44.285201 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 01:03:44.285201 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 01:03:44.285201 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 01:03:44.285201 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 01:03:44.285201 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 01:03:44.285201 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 01:03:44.285201 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Aug 31 01:03:44.289205 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 138/261] 10.222.0.20 () {56 vars in 1114 bytes} [Wed Aug 31 01:03:44 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:03:44.291507 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23731a25-cb69-45b8-8414-6cfa1739e0bf None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:44.380894 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:44.386862 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e5a1b7-29bf-4c64-8225-187e06f8245e None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:44.418548 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23731a25-cb69-45b8-8414-6cfa1739e0bf None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:44.458180 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c2bb49-bcf6-4bdb-9fa4-646da49e69d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:44.459035 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c2bb49-bcf6-4bdb-9fa4-646da49e69d3 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:44.459489 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c2bb49-bcf6-4bdb-9fa4-646da49e69d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:44.459981 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c2bb49-bcf6-4bdb-9fa4-646da49e69d3 None admin] PATH_INFO: `/v3/roles` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:44.460769 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa353525-eabe-4467-9440-eaf9a3131e99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:44.460978 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c2bb49-bcf6-4bdb-9fa4-646da49e69d3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:03:44.461436 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c2bb49-bcf6-4bdb-9fa4-646da49e69d3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:44.461677 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa353525-eabe-4467-9440-eaf9a3131e99 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:44.462566 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa353525-eabe-4467-9440-eaf9a3131e99 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:44.462946 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa353525-eabe-4467-9440-eaf9a3131e99 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:44.463940 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c2bb49-bcf6-4bdb-9fa4-646da49e69d3 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:44.474366 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 139/262] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Aug 31 01:03:44 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:03:44.481592 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c2bb49-bcf6-4bdb-9fa4-646da49e69d3 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:44.518969 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:44.602220 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c2bb49-bcf6-4bdb-9fa4-646da49e69d3 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:44.616591 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa353525-eabe-4467-9440-eaf9a3131e99 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:44.618241 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa353525-eabe-4467-9440-eaf9a3131e99 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:44.639131 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6f0ee9-3024-479c-ac00-40ff86b110a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:44.640005 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6f0ee9-3024-479c-ac00-40ff86b110a2 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:44.640430 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6f0ee9-3024-479c-ac00-40ff86b110a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:44.640882 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6f0ee9-3024-479c-ac00-40ff86b110a2 None admin] PATH_INFO: `/v3/users/9b7f934ca2db4023b1d3e2aff511e838` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:44.642212 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6f0ee9-3024-479c-ac00-40ff86b110a2 None admin] RBAC: Authorizing `identity:get_user(user_id=9b7f934ca2db4023b1d3e2aff511e838)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:44.643880 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6f0ee9-3024-479c-ac00-40ff86b110a2 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:44.645929 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 140/263] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Aug 31 01:03:44 2022] GET /identity/v3/users/9b7f934ca2db4023b1d3e2aff511e838 => generated 283 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 01:03:44.652150 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6f0ee9-3024-479c-ac00-40ff86b110a2 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:44.756701 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6f0ee9-3024-479c-ac00-40ff86b110a2 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:44.780967 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:44.801108 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda1639f-8a96-4d26-bd0e-7ac3c9a16101 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:44.801970 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda1639f-8a96-4d26-bd0e-7ac3c9a16101 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:44.802402 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda1639f-8a96-4d26-bd0e-7ac3c9a16101 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:44.802821 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda1639f-8a96-4d26-bd0e-7ac3c9a16101 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:44.816469 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:44.829282 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 124/264] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Aug 31 01:03:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 725 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 01:03:44.834855 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa353525-eabe-4467-9440-eaf9a3131e99 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:44.922279 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda1639f-8a96-4d26-bd0e-7ac3c9a16101 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:44.924023 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda1639f-8a96-4d26-bd0e-7ac3c9a16101 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:45.018891 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa353525-eabe-4467-9440-eaf9a3131e99 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:45.077712 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab5029f-5f4d-4eee-aea7-e298767371c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:45.078355 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab5029f-5f4d-4eee-aea7-e298767371c5 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:45.078635 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab5029f-5f4d-4eee-aea7-e298767371c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:45.078911 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab5029f-5f4d-4eee-aea7-e298767371c5 None admin] PATH_INFO: `/v3/projects/b4ae9d03e35549cfbadf1d1001837bb6` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:45.085429 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab5029f-5f4d-4eee-aea7-e298767371c5 None admin] RBAC: Authorizing `identity:get_project(project_id=b4ae9d03e35549cfbadf1d1001837bb6)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:45.087164 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab5029f-5f4d-4eee-aea7-e298767371c5 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:45.089185 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 125/265] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Aug 31 01:03:44 2022] GET /identity/v3/projects/b4ae9d03e35549cfbadf1d1001837bb6 => generated 302 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:03:45.095035 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab5029f-5f4d-4eee-aea7-e298767371c5 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:45.103158 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:45.107407 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 141/266] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Aug 31 01:03:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 455 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 31 01:03:45.113670 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda1639f-8a96-4d26-bd0e-7ac3c9a16101 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:45.205806 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:45.282894 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab5029f-5f4d-4eee-aea7-e298767371c5 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:45.310646 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda1639f-8a96-4d26-bd0e-7ac3c9a16101 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:45.350666 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5404d1-f557-408e-8d6e-1f8c52fc4927 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:45.351633 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5404d1-f557-408e-8d6e-1f8c52fc4927 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:45.352137 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5404d1-f557-408e-8d6e-1f8c52fc4927 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:45.352624 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5404d1-f557-408e-8d6e-1f8c52fc4927 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:45.354392 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5404d1-f557-408e-8d6e-1f8c52fc4927 None admin] RBAC: Adding query filter params (role.id=d009411496b8452b88e77ffb8db015a7, scope.project.id=b4ae9d03e35549cfbadf1d1001837bb6, user.id=9b7f934ca2db4023b1d3e2aff511e838) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:03:45.354923 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5404d1-f557-408e-8d6e-1f8c52fc4927 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:45.356563 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5404d1-f557-408e-8d6e-1f8c52fc4927 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:45.372465 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 126/267] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Aug 31 01:03:45 2022] GET /identity/v3/role_assignments?user.id=9b7f934ca2db4023b1d3e2aff511e838&scope.project.id=b4ae9d03e35549cfbadf1d1001837bb6&role.id=d009411496b8452b88e77ffb8db015a7 => generated 608 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 01:03:45.409659 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9a3b07-8fd4-40cd-906e-feae1357adfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:45.410412 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9a3b07-8fd4-40cd-906e-feae1357adfb None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:45.410696 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9a3b07-8fd4-40cd-906e-feae1357adfb None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:45.410975 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9a3b07-8fd4-40cd-906e-feae1357adfb None admin] PATH_INFO: `/v3/projects/b4ae9d03e35549cfbadf1d1001837bb6` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:45.412339 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9a3b07-8fd4-40cd-906e-feae1357adfb None admin] RBAC: Authorizing `identity:get_project(project_id=b4ae9d03e35549cfbadf1d1001837bb6)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:45.413875 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9a3b07-8fd4-40cd-906e-feae1357adfb None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:45.416077 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 142/268] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Aug 31 01:03:45 2022] GET /identity/v3/projects/b4ae9d03e35549cfbadf1d1001837bb6 => generated 302 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:03:45.422512 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5404d1-f557-408e-8d6e-1f8c52fc4927 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:45.550805 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5404d1-f557-408e-8d6e-1f8c52fc4927 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:45.593356 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7022b01-b699-4eb0-a39c-e05dce8720ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:45.594387 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:45.594909 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7022b01-b699-4eb0-a39c-e05dce8720ed None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:45.595419 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7022b01-b699-4eb0-a39c-e05dce8720ed None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:45.595892 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7022b01-b699-4eb0-a39c-e05dce8720ed None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:45.596896 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7022b01-b699-4eb0-a39c-e05dce8720ed None admin] RBAC: Adding query filter params (role.id=bebc77e8dfd54dbba7079f93218e7c7d, scope.project.id=b4ae9d03e35549cfbadf1d1001837bb6, user.id=9b7f934ca2db4023b1d3e2aff511e838) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:03:45.597422 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7022b01-b699-4eb0-a39c-e05dce8720ed None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:45.599089 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7022b01-b699-4eb0-a39c-e05dce8720ed None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:45.605537 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 127/269] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Aug 31 01:03:45 2022] GET /identity/v3/role_assignments?user.id=9b7f934ca2db4023b1d3e2aff511e838&scope.project.id=b4ae9d03e35549cfbadf1d1001837bb6&role.id=bebc77e8dfd54dbba7079f93218e7c7d => generated 608 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:03:45.700953 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07750aae-1424-4428-987f-170a21396085 None None] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:45.701494 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07750aae-1424-4428-987f-170a21396085 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:45.701494 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07750aae-1424-4428-987f-170a21396085 None None] PATH_INFO: `/` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:45.703076 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 143/270] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:03:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:03:45.708805 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0d88dc-d9dd-40b6-8f1e-7b404284bcdc None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:45.709369 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0d88dc-d9dd-40b6-8f1e-7b404284bcdc None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:45.709851 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0d88dc-d9dd-40b6-8f1e-7b404284bcdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:45.735838 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-5e0d88dc-d9dd-40b6-8f1e-7b404284bcdc None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:03:45.741836 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-5e0d88dc-d9dd-40b6-8f1e-7b404284bcdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:03:45.788816 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bc8b16-3138-4d85-a5ca-32047e40975e None None] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:45.789388 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bc8b16-3138-4d85-a5ca-32047e40975e None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:45.789873 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bc8b16-3138-4d85-a5ca-32047e40975e None None] PATH_INFO: `/` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:45.791474 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 144/271] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:03:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:03:45.797060 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffbc205-8993-4561-8dbc-505527882420 None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:45.797599 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffbc205-8993-4561-8dbc-505527882420 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:45.798091 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffbc205-8993-4561-8dbc-505527882420 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:45.833755 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-0ffbc205-8993-4561-8dbc-505527882420 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:03:45.835262 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-0ffbc205-8993-4561-8dbc-505527882420 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:03:45.863525 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 128/272] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:03:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 01:03:45.929759 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:45.936374 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 145/273] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:03:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 01:03:45.989857 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f805a423-ae71-46f0-ad0c-cbb0ef29fdcc None None] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:45.990475 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f805a423-ae71-46f0-ad0c-cbb0ef29fdcc None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:45.990950 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f805a423-ae71-46f0-ad0c-cbb0ef29fdcc None None] PATH_INFO: `/` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:45.992797 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 129/274] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:03:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:03:45.998754 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26590bc5-a6ce-495d-a652-cbe3d34eb255 None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:45.999051 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26590bc5-a6ce-495d-a652-cbe3d34eb255 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:45.999332 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26590bc5-a6ce-495d-a652-cbe3d34eb255 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:46.019948 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-26590bc5-a6ce-495d-a652-cbe3d34eb255 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:03:46.021374 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-26590bc5-a6ce-495d-a652-cbe3d34eb255 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:03:46.098027 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 146/275] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:03:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 01:03:46.178172 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f805a423-ae71-46f0-ad0c-cbb0ef29fdcc None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:46.245985 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26590bc5-a6ce-495d-a652-cbe3d34eb255 None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:46.305101 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f805a423-ae71-46f0-ad0c-cbb0ef29fdcc None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:46.346079 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:46.360490 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26590bc5-a6ce-495d-a652-cbe3d34eb255 None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:46.362347 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3b18bf-c590-4db8-8f1b-7da19f836b43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:46.362984 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3b18bf-c590-4db8-8f1b-7da19f836b43 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:46.363264 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3b18bf-c590-4db8-8f1b-7da19f836b43 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:46.363539 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3b18bf-c590-4db8-8f1b-7da19f836b43 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:46.378734 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3b18bf-c590-4db8-8f1b-7da19f836b43 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 01:03:46.385696 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application [None req-be3b18bf-c590-4db8-8f1b-7da19f836b43 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 01:03:46.385696 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:03:46.385696 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:03:46.385696 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:03:46.385696 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:03:46.385696 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:03:46.385696 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:03:46.385696 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:03:46.385696 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:03:46.385696 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:03:46.385696 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:03:46.385696 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:03:46.385696 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 01:03:46.385696 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 01:03:46.385696 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 01:03:46.385696 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 01:03:46.386741 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 01:03:46.386741 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 01:03:46.386741 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 01:03:46.386741 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 01:03:46.386741 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:03:46.386741 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:03:46.386741 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:03:46.386741 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:03:46.386741 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:03:46.386741 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:03:46.386741 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:03:46.386741 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application with Lock( Aug 31 01:03:46.386741 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:03:46.386741 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:03:46.386741 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:03:46.386741 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:03:46.387717 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:03:46.387717 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:03:46.387717 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:03:46.387717 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:03:46.387717 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 01:03:46.387717 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 01:03:46.387717 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 01:03:46.387717 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 01:03:46.387717 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 01:03:46.387717 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 01:03:46.387717 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 01:03:46.387717 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Aug 31 01:03:46.390155 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 130/276] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Aug 31 01:03:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:03:46.396130 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3b18bf-c590-4db8-8f1b-7da19f836b43 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:46.398466 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:46.408440 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f35f92-abd1-46ad-853e-faa02b36f421 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:46.410823 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f35f92-abd1-46ad-853e-faa02b36f421 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:46.410823 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f35f92-abd1-46ad-853e-faa02b36f421 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:46.410823 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f35f92-abd1-46ad-853e-faa02b36f421 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:46.424001 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f35f92-abd1-46ad-853e-faa02b36f421 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 01:03:46.431370 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application [None req-e7f35f92-abd1-46ad-853e-faa02b36f421 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 01:03:46.431370 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:03:46.431370 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:03:46.431370 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:03:46.431370 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:03:46.431370 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:03:46.431370 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:03:46.431370 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:03:46.431370 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:03:46.431370 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:03:46.431370 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:03:46.431370 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:03:46.431370 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 01:03:46.431370 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 01:03:46.431370 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 01:03:46.431370 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 01:03:46.432335 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 01:03:46.432335 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 01:03:46.432335 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 01:03:46.432335 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 01:03:46.432335 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:03:46.432335 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:03:46.432335 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:03:46.432335 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:03:46.432335 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:03:46.432335 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:03:46.432335 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:03:46.432335 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application with Lock( Aug 31 01:03:46.432335 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:03:46.432335 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:03:46.432335 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:03:46.432335 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:03:46.433253 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:03:46.433253 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:03:46.433253 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:03:46.433253 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:03:46.433253 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 01:03:46.433253 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 01:03:46.433253 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 01:03:46.433253 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 01:03:46.433253 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 01:03:46.433253 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 01:03:46.433253 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 01:03:46.433253 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Aug 31 01:03:46.436865 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 147/277] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Aug 31 01:03:46 2022] GET /identity/v3/roles/reader => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:03:46.437713 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f35f92-abd1-46ad-853e-faa02b36f421 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:46.526928 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3b18bf-c590-4db8-8f1b-7da19f836b43 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:46.541009 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f35f92-abd1-46ad-853e-faa02b36f421 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:46.565933 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2aed5b4-7b2c-4770-ba95-c1fdca5afbb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:46.566801 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2aed5b4-7b2c-4770-ba95-c1fdca5afbb8 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:46.567236 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2aed5b4-7b2c-4770-ba95-c1fdca5afbb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:46.567513 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2aed5b4-7b2c-4770-ba95-c1fdca5afbb8 None admin] PATH_INFO: `/v3/roles` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:46.568175 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2aed5b4-7b2c-4770-ba95-c1fdca5afbb8 None admin] RBAC: Adding query filter params (name=member) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:03:46.568479 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2aed5b4-7b2c-4770-ba95-c1fdca5afbb8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:46.569918 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2aed5b4-7b2c-4770-ba95-c1fdca5afbb8 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:46.576732 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d24fef-6002-4f7c-9cec-f3f81ff8eca1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:46.577373 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d24fef-6002-4f7c-9cec-f3f81ff8eca1 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:46.577657 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d24fef-6002-4f7c-9cec-f3f81ff8eca1 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:46.577928 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d24fef-6002-4f7c-9cec-f3f81ff8eca1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:46.578928 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 131/278] 10.222.0.20 () {56 vars in 1101 bytes} [Wed Aug 31 01:03:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 01:03:46.579522 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2aed5b4-7b2c-4770-ba95-c1fdca5afbb8 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:46.583756 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d24fef-6002-4f7c-9cec-f3f81ff8eca1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:46.585008 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d24fef-6002-4f7c-9cec-f3f81ff8eca1 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:46.588858 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 148/279] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 01:03:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:03:46.588858 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d24fef-6002-4f7c-9cec-f3f81ff8eca1 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:46.636831 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:46.646773 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:46.689395 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2aed5b4-7b2c-4770-ba95-c1fdca5afbb8 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:46.689664 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d24fef-6002-4f7c-9cec-f3f81ff8eca1 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:46.724912 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddb2fc9-a421-46b7-9711-1db4537a0047 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:46.725528 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddb2fc9-a421-46b7-9711-1db4537a0047 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:46.725812 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddb2fc9-a421-46b7-9711-1db4537a0047 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:46.726091 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddb2fc9-a421-46b7-9711-1db4537a0047 None admin] PATH_INFO: `/v3/users/f0979995b04449ba816ce361cc47a953` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:46.727196 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400b524a-0a45-409e-bca3-7bd57b5f8b2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:46.728079 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400b524a-0a45-409e-bca3-7bd57b5f8b2d None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:46.728547 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400b524a-0a45-409e-bca3-7bd57b5f8b2d None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:46.729044 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400b524a-0a45-409e-bca3-7bd57b5f8b2d None admin] PATH_INFO: `/v3/roles` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:46.729775 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400b524a-0a45-409e-bca3-7bd57b5f8b2d None admin] RBAC: Adding query filter params (name=reader) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:03:46.730086 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400b524a-0a45-409e-bca3-7bd57b5f8b2d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:46.731460 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400b524a-0a45-409e-bca3-7bd57b5f8b2d None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:46.739676 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddb2fc9-a421-46b7-9711-1db4537a0047 None admin] RBAC: Authorizing `identity:get_user(user_id=f0979995b04449ba816ce361cc47a953)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:46.745251 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 132/280] 10.222.0.20 () {56 vars in 1101 bytes} [Wed Aug 31 01:03:46 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:03:46.745251 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400b524a-0a45-409e-bca3-7bd57b5f8b2d None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:46.745467 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddb2fc9-a421-46b7-9711-1db4537a0047 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:46.745467 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 149/281] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Aug 31 01:03:46 2022] GET /identity/v3/users/f0979995b04449ba816ce361cc47a953 => generated 293 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:03:46.748918 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddb2fc9-a421-46b7-9711-1db4537a0047 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:46.848583 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400b524a-0a45-409e-bca3-7bd57b5f8b2d None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:46.849252 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddb2fc9-a421-46b7-9711-1db4537a0047 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:46.859174 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:46.862956 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:46.881862 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af4bbd9-69dd-44f3-a703-f9c5f8e82fc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:46.882469 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af4bbd9-69dd-44f3-a703-f9c5f8e82fc7 None admin] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:46.882572 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e08b079-1deb-4fd0-8a9d-87e1df2454df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:46.882752 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af4bbd9-69dd-44f3-a703-f9c5f8e82fc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:46.883024 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af4bbd9-69dd-44f3-a703-f9c5f8e82fc7 None admin] PATH_INFO: `/v3/users` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:46.883111 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e08b079-1deb-4fd0-8a9d-87e1df2454df None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:46.883381 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e08b079-1deb-4fd0-8a9d-87e1df2454df None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:46.883660 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e08b079-1deb-4fd0-8a9d-87e1df2454df None admin] PATH_INFO: `/v3/users/49f42887f51c497d8ae8ff3f7eb71e07` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:46.883744 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af4bbd9-69dd-44f3-a703-f9c5f8e82fc7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:46.884850 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e08b079-1deb-4fd0-8a9d-87e1df2454df None admin] RBAC: Authorizing `identity:get_user(user_id=49f42887f51c497d8ae8ff3f7eb71e07)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:46.885171 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af4bbd9-69dd-44f3-a703-f9c5f8e82fc7 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:46.886328 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e08b079-1deb-4fd0-8a9d-87e1df2454df None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:46.887963 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 150/282] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Aug 31 01:03:46 2022] GET /identity/v3/users/49f42887f51c497d8ae8ff3f7eb71e07 => generated 297 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:03:46.888672 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e08b079-1deb-4fd0-8a9d-87e1df2454df None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:46.985110 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e08b079-1deb-4fd0-8a9d-87e1df2454df None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:47.006264 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 133/283] 10.222.0.20 () {60 vars in 1132 bytes} [Wed Aug 31 01:03:46 2022] POST /identity/v3/users => generated 293 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:03:47.007546 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c85265-dcca-4656-8a85-3d2483fa8098 None None] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:47.007844 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c85265-dcca-4656-8a85-3d2483fa8098 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:47.008121 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c85265-dcca-4656-8a85-3d2483fa8098 None None] PATH_INFO: `/` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:47.012777 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 134/284] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:03:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:03:47.012777 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c85265-dcca-4656-8a85-3d2483fa8098 None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:47.023659 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2922fa2-bd2d-48a0-b1c2-d331b83bb68b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:47.024335 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2922fa2-bd2d-48a0-b1c2-d331b83bb68b None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:47.024621 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2922fa2-bd2d-48a0-b1c2-d331b83bb68b None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:47.024934 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2922fa2-bd2d-48a0-b1c2-d331b83bb68b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:47.025679 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2922fa2-bd2d-48a0-b1c2-d331b83bb68b None admin] RBAC: Adding query filter params (role.id=d0b9768f4d014766b9326387f2e11f34, user.id=f0979995b04449ba816ce361cc47a953, scope.system=all) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:03:47.025994 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2922fa2-bd2d-48a0-b1c2-d331b83bb68b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:47.027530 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2922fa2-bd2d-48a0-b1c2-d331b83bb68b None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:47.042525 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 151/285] 10.222.0.20 () {56 vars in 1308 bytes} [Wed Aug 31 01:03:46 2022] GET /identity/v3/role_assignments?user.id=f0979995b04449ba816ce361cc47a953&scope.system=all&role.id=d0b9768f4d014766b9326387f2e11f34 => generated 226 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:03:47.042525 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e6b5a4-e2cb-4508-82e1-5bc93a30a397 None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:47.042525 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e6b5a4-e2cb-4508-82e1-5bc93a30a397 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:47.042525 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e6b5a4-e2cb-4508-82e1-5bc93a30a397 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:47.071174 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-c9e6b5a4-e2cb-4508-82e1-5bc93a30a397 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:03:47.073435 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-c9e6b5a4-e2cb-4508-82e1-5bc93a30a397 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:03:47.146736 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:47.151693 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c85265-dcca-4656-8a85-3d2483fa8098 None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:47.185747 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 152/286] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:03:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 01:03:47.199174 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132977c8-300b-44d2-a3d5-499428b2f709 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:47.199859 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132977c8-300b-44d2-a3d5-499428b2f709 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:47.200135 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132977c8-300b-44d2-a3d5-499428b2f709 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:47.200463 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132977c8-300b-44d2-a3d5-499428b2f709 None admin] PATH_INFO: `/v3/projects/b4ae9d03e35549cfbadf1d1001837bb6` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:47.201702 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132977c8-300b-44d2-a3d5-499428b2f709 None admin] RBAC: Authorizing `identity:get_project(project_id=b4ae9d03e35549cfbadf1d1001837bb6)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:47.203255 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132977c8-300b-44d2-a3d5-499428b2f709 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:47.204950 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 135/287] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Aug 31 01:03:47 2022] GET /identity/v3/projects/b4ae9d03e35549cfbadf1d1001837bb6 => generated 302 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:03:47.210840 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e6b5a4-e2cb-4508-82e1-5bc93a30a397 None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:47.315612 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e6b5a4-e2cb-4508-82e1-5bc93a30a397 None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:47.350600 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea070f89-d933-4074-aaab-18e209aa99fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:47.351272 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea070f89-d933-4074-aaab-18e209aa99fd None admin] REQUEST_METHOD: `PUT` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:47.351543 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea070f89-d933-4074-aaab-18e209aa99fd None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:47.351817 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea070f89-d933-4074-aaab-18e209aa99fd None admin] PATH_INFO: `/v3/projects/b4ae9d03e35549cfbadf1d1001837bb6/users/49f42887f51c497d8ae8ff3f7eb71e07/roles/e1aef2df18ab425a8b78190b0fbfe8ba` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:47.354365 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea070f89-d933-4074-aaab-18e209aa99fd None admin] RBAC: Authorizing `identity:create_grant(project_id=b4ae9d03e35549cfbadf1d1001837bb6, user_id=49f42887f51c497d8ae8ff3f7eb71e07, role_id=e1aef2df18ab425a8b78190b0fbfe8ba)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:47.355805 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea070f89-d933-4074-aaab-18e209aa99fd None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:47.437229 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 153/288] 10.222.0.20 () {58 vars in 1439 bytes} [Wed Aug 31 01:03:47 2022] PUT /identity/v3/projects/b4ae9d03e35549cfbadf1d1001837bb6/users/49f42887f51c497d8ae8ff3f7eb71e07/roles/e1aef2df18ab425a8b78190b0fbfe8ba => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:03:47.465715 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132977c8-300b-44d2-a3d5-499428b2f709 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:47.507716 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:47.593233 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132977c8-300b-44d2-a3d5-499428b2f709 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:47.629830 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df033a39-c2c4-41b1-9426-347aa4807759 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:47.630481 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df033a39-c2c4-41b1-9426-347aa4807759 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:47.630811 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df033a39-c2c4-41b1-9426-347aa4807759 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:47.631098 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df033a39-c2c4-41b1-9426-347aa4807759 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:47.642767 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df033a39-c2c4-41b1-9426-347aa4807759 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 01:03:47.648326 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application [None req-df033a39-c2c4-41b1-9426-347aa4807759 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 01:03:47.648326 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:03:47.648326 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:03:47.648326 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:03:47.648326 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:03:47.648326 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:03:47.648326 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:03:47.648326 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:03:47.648326 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:03:47.648326 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:03:47.648326 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:03:47.648326 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:03:47.648326 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 01:03:47.648326 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 01:03:47.648326 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 01:03:47.648326 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 01:03:47.649260 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 01:03:47.649260 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 01:03:47.649260 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 01:03:47.649260 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 01:03:47.649260 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:03:47.649260 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:03:47.649260 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:03:47.649260 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:03:47.649260 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:03:47.649260 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:03:47.649260 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:03:47.649260 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application with Lock( Aug 31 01:03:47.649260 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:03:47.649260 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:03:47.649260 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:03:47.649260 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:03:47.650633 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:03:47.650633 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:03:47.650633 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:03:47.650633 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:03:47.650633 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 01:03:47.650633 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 01:03:47.650633 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 01:03:47.650633 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 01:03:47.650633 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 01:03:47.650633 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 01:03:47.650633 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 01:03:47.650633 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Aug 31 01:03:47.650633 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 136/289] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Aug 31 01:03:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:03:47.656015 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea070f89-d933-4074-aaab-18e209aa99fd None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:47.692516 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:47.763056 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea070f89-d933-4074-aaab-18e209aa99fd None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:47.798862 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3662720-27d0-46aa-9028-665fc0d776cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:47.799477 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3662720-27d0-46aa-9028-665fc0d776cb None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:47.799763 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3662720-27d0-46aa-9028-665fc0d776cb None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:47.800043 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3662720-27d0-46aa-9028-665fc0d776cb None admin] PATH_INFO: `/v3/roles` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:47.800706 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3662720-27d0-46aa-9028-665fc0d776cb None admin] RBAC: Adding query filter params (name=member) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:03:47.801044 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3662720-27d0-46aa-9028-665fc0d776cb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:47.802412 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3662720-27d0-46aa-9028-665fc0d776cb None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:47.811443 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 154/290] 10.222.0.20 () {56 vars in 1101 bytes} [Wed Aug 31 01:03:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:03:47.817110 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df033a39-c2c4-41b1-9426-347aa4807759 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:47.941851 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df033a39-c2c4-41b1-9426-347aa4807759 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:47.945141 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:47.984353 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209caf6c-80b3-4d01-b355-a798bcdc1434 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:47.985084 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209caf6c-80b3-4d01-b355-a798bcdc1434 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:47.985371 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209caf6c-80b3-4d01-b355-a798bcdc1434 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:47.985703 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209caf6c-80b3-4d01-b355-a798bcdc1434 None admin] PATH_INFO: `/v3/users/22434d0e26cd419cb313508b0d0ebd7e` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:47.986870 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209caf6c-80b3-4d01-b355-a798bcdc1434 None admin] RBAC: Authorizing `identity:get_user(user_id=22434d0e26cd419cb313508b0d0ebd7e)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:47.988346 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209caf6c-80b3-4d01-b355-a798bcdc1434 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:47.989964 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 137/291] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Aug 31 01:03:47 2022] GET /identity/v3/users/22434d0e26cd419cb313508b0d0ebd7e => generated 297 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:03:47.996199 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3662720-27d0-46aa-9028-665fc0d776cb None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:48.100713 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:48.105288 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3662720-27d0-46aa-9028-665fc0d776cb None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:48.139987 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834120d4-8efc-4998-a868-707c470faddb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:48.140650 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834120d4-8efc-4998-a868-707c470faddb None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:48.140966 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834120d4-8efc-4998-a868-707c470faddb None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:48.141244 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834120d4-8efc-4998-a868-707c470faddb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:48.211774 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834120d4-8efc-4998-a868-707c470faddb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:48.213783 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834120d4-8efc-4998-a868-707c470faddb None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:48.313268 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:48.318093 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 155/292] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Aug 31 01:03:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 01:03:48.328841 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209caf6c-80b3-4d01-b355-a798bcdc1434 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:48.444518 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209caf6c-80b3-4d01-b355-a798bcdc1434 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:48.484281 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61fa2c1-cfaf-47eb-8b4a-1f778c5837d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:48.484882 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61fa2c1-cfaf-47eb-8b4a-1f778c5837d5 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:48.485171 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61fa2c1-cfaf-47eb-8b4a-1f778c5837d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:48.485454 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61fa2c1-cfaf-47eb-8b4a-1f778c5837d5 None admin] PATH_INFO: `/v3/projects/b4ae9d03e35549cfbadf1d1001837bb6` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:48.486593 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61fa2c1-cfaf-47eb-8b4a-1f778c5837d5 None admin] RBAC: Authorizing `identity:get_project(project_id=b4ae9d03e35549cfbadf1d1001837bb6)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:48.488051 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61fa2c1-cfaf-47eb-8b4a-1f778c5837d5 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:48.489890 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 138/293] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Aug 31 01:03:48 2022] GET /identity/v3/projects/b4ae9d03e35549cfbadf1d1001837bb6 => generated 302 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:03:48.495631 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834120d4-8efc-4998-a868-707c470faddb None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:48.611516 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834120d4-8efc-4998-a868-707c470faddb None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:48.654394 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd35bce-223e-4691-b485-3c0c23ab2010 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:48.655025 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd35bce-223e-4691-b485-3c0c23ab2010 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:48.655304 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd35bce-223e-4691-b485-3c0c23ab2010 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:48.655576 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd35bce-223e-4691-b485-3c0c23ab2010 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:48.656303 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd35bce-223e-4691-b485-3c0c23ab2010 None admin] RBAC: Adding query filter params (role.id=d0b9768f4d014766b9326387f2e11f34, scope.project.id=b4ae9d03e35549cfbadf1d1001837bb6, user.id=22434d0e26cd419cb313508b0d0ebd7e) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:03:48.656611 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd35bce-223e-4691-b485-3c0c23ab2010 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:48.658086 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd35bce-223e-4691-b485-3c0c23ab2010 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:48.669622 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 156/294] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Aug 31 01:03:48 2022] GET /identity/v3/role_assignments?user.id=22434d0e26cd419cb313508b0d0ebd7e&scope.project.id=b4ae9d03e35549cfbadf1d1001837bb6&role.id=d0b9768f4d014766b9326387f2e11f34 => generated 608 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:03:49.226607 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ee7d7b-2c01-4d0c-9212-3c22c9fcdf2c None None] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:49.227113 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ee7d7b-2c01-4d0c-9212-3c22c9fcdf2c None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:49.227187 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ee7d7b-2c01-4d0c-9212-3c22c9fcdf2c None None] PATH_INFO: `/` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:49.228911 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 139/295] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:03:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:03:49.233836 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1daec7-f49b-42d1-9445-14d17670f880 None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:49.234253 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1daec7-f49b-42d1-9445-14d17670f880 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:49.234422 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1daec7-f49b-42d1-9445-14d17670f880 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:49.257646 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-5e1daec7-f49b-42d1-9445-14d17670f880 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:03:49.259374 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-5e1daec7-f49b-42d1-9445-14d17670f880 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:03:49.322638 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:49.334100 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 157/296] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:03:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 01:03:49.534807 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59504485-7ec4-4629-a770-4523c55d4d02 None None] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:49.535134 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59504485-7ec4-4629-a770-4523c55d4d02 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:49.535312 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59504485-7ec4-4629-a770-4523c55d4d02 None None] PATH_INFO: `/` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:49.536868 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 140/297] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:03:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:03:49.542195 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1306049a-3c3e-47c1-82e6-275fea5537bd None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:49.542484 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1306049a-3c3e-47c1-82e6-275fea5537bd None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:49.542756 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1306049a-3c3e-47c1-82e6-275fea5537bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:49.561364 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-1306049a-3c3e-47c1-82e6-275fea5537bd None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:03:49.562547 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-1306049a-3c3e-47c1-82e6-275fea5537bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:03:49.613000 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59504485-7ec4-4629-a770-4523c55d4d02 None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:49.629684 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 158/298] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:03:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 01:03:49.636667 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a15155-6b1e-4a34-8230-806d32c5342e None None] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:49.637151 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a15155-6b1e-4a34-8230-806d32c5342e None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:49.637603 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a15155-6b1e-4a34-8230-806d32c5342e None None] PATH_INFO: `/` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:49.639357 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 159/299] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:03:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 01:03:49.644397 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acdf7f3-2ac3-4d4c-855d-d5fbc3543fe4 None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:49.644987 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:49.645371 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acdf7f3-2ac3-4d4c-855d-d5fbc3543fe4 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:49.645660 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acdf7f3-2ac3-4d4c-855d-d5fbc3543fe4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:49.664155 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-3acdf7f3-2ac3-4d4c-855d-d5fbc3543fe4 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:03:49.665484 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-3acdf7f3-2ac3-4d4c-855d-d5fbc3543fe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:03:49.725499 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59504485-7ec4-4629-a770-4523c55d4d02 None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:49.748548 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 160/300] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:03:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 01:03:49.767596 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072036cc-cfbc-4a46-967f-ada9d235bd52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:49.768382 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072036cc-cfbc-4a46-967f-ada9d235bd52 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:49.768673 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072036cc-cfbc-4a46-967f-ada9d235bd52 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:49.768994 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072036cc-cfbc-4a46-967f-ada9d235bd52 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:49.780241 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072036cc-cfbc-4a46-967f-ada9d235bd52 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 01:03:49.785313 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application [None req-072036cc-cfbc-4a46-967f-ada9d235bd52 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 01:03:49.785313 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:03:49.785313 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:03:49.785313 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:03:49.785313 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:03:49.785313 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:03:49.785313 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:03:49.785313 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:03:49.785313 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:03:49.785313 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:03:49.785313 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:03:49.785313 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:03:49.785313 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 01:03:49.785313 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 01:03:49.785313 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 01:03:49.785313 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 01:03:49.786191 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 01:03:49.786191 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 01:03:49.786191 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 01:03:49.786191 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 01:03:49.786191 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:03:49.786191 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:03:49.786191 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:03:49.786191 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:03:49.786191 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:03:49.786191 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:03:49.786191 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:03:49.786191 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application with Lock( Aug 31 01:03:49.786191 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:03:49.786191 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:03:49.786191 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:03:49.786191 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:03:49.786975 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:03:49.786975 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:03:49.786975 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:03:49.786975 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:03:49.786975 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 01:03:49.786975 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 01:03:49.786975 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 01:03:49.786975 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 01:03:49.786975 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 01:03:49.786975 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 01:03:49.786975 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 01:03:49.786975 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Aug 31 01:03:49.786975 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 141/301] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Aug 31 01:03:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:03:49.792801 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acdf7f3-2ac3-4d4c-855d-d5fbc3543fe4 None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:49.848552 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:49.907542 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acdf7f3-2ac3-4d4c-855d-d5fbc3543fe4 None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:49.938189 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072036cc-cfbc-4a46-967f-ada9d235bd52 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:49.944673 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65822f00-0bd1-48e9-8dbe-db4fde9b56e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:49.945323 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65822f00-0bd1-48e9-8dbe-db4fde9b56e8 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:49.945608 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65822f00-0bd1-48e9-8dbe-db4fde9b56e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:49.945882 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65822f00-0bd1-48e9-8dbe-db4fde9b56e8 None admin] PATH_INFO: `/v3/roles` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:49.947071 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65822f00-0bd1-48e9-8dbe-db4fde9b56e8 None admin] RBAC: Adding query filter params (name=member) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:03:49.947379 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65822f00-0bd1-48e9-8dbe-db4fde9b56e8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:49.949600 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65822f00-0bd1-48e9-8dbe-db4fde9b56e8 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:49.961349 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 161/302] 10.222.0.20 () {56 vars in 1101 bytes} [Wed Aug 31 01:03:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:03:49.967334 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65822f00-0bd1-48e9-8dbe-db4fde9b56e8 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:50.038634 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:50.047798 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072036cc-cfbc-4a46-967f-ada9d235bd52 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:50.067375 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65822f00-0bd1-48e9-8dbe-db4fde9b56e8 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:50.081535 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:50.082294 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d489e555-b349-4876-b62b-9453ccc8b49d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:50.082921 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d489e555-b349-4876-b62b-9453ccc8b49d None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:50.083202 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d489e555-b349-4876-b62b-9453ccc8b49d None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:50.083475 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d489e555-b349-4876-b62b-9453ccc8b49d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:50.094192 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d489e555-b349-4876-b62b-9453ccc8b49d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 01:03:50.099096 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application [None req-d489e555-b349-4876-b62b-9453ccc8b49d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 01:03:50.099096 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:03:50.099096 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:03:50.099096 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:03:50.099096 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:03:50.099096 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:03:50.099096 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:03:50.099096 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:03:50.099096 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:03:50.099096 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:03:50.099096 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:03:50.099096 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:03:50.099096 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 01:03:50.099096 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 01:03:50.099096 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 01:03:50.099096 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 01:03:50.105735 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 01:03:50.105735 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 01:03:50.105735 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 01:03:50.105735 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 01:03:50.105735 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:03:50.105735 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:03:50.105735 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:03:50.105735 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:03:50.105735 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:03:50.105735 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:03:50.105735 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:03:50.105735 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application with Lock( Aug 31 01:03:50.105735 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:03:50.105735 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:03:50.105735 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:03:50.105735 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:03:50.106646 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:03:50.106646 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:03:50.106646 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:03:50.106646 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:03:50.106646 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 01:03:50.106646 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 01:03:50.106646 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 01:03:50.106646 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 01:03:50.106646 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 01:03:50.106646 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 01:03:50.106646 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 01:03:50.106646 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Aug 31 01:03:50.106646 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 142/303] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Aug 31 01:03:49 2022] GET /identity/v3/roles/reader => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 01:03:50.107519 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93212f09-d7f8-4a5e-894f-18d2d8946c07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:50.107605 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972d59e5-7ba0-4177-bec4-1762277e4cbf None None] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:50.107605 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972d59e5-7ba0-4177-bec4-1762277e4cbf None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:50.107721 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93212f09-d7f8-4a5e-894f-18d2d8946c07 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:50.107794 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972d59e5-7ba0-4177-bec4-1762277e4cbf None None] PATH_INFO: `/` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:50.107860 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93212f09-d7f8-4a5e-894f-18d2d8946c07 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:50.107860 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93212f09-d7f8-4a5e-894f-18d2d8946c07 None admin] PATH_INFO: `/v3/users/f0979995b04449ba816ce361cc47a953` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:50.107985 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 143/304] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:03:50 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:03:50.107985 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972d59e5-7ba0-4177-bec4-1762277e4cbf None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:50.108106 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93212f09-d7f8-4a5e-894f-18d2d8946c07 None admin] RBAC: Authorizing `identity:get_user(user_id=f0979995b04449ba816ce361cc47a953)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:50.108106 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93212f09-d7f8-4a5e-894f-18d2d8946c07 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:50.109197 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 162/305] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Aug 31 01:03:49 2022] GET /identity/v3/users/f0979995b04449ba816ce361cc47a953 => generated 293 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:03:50.110284 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93212f09-d7f8-4a5e-894f-18d2d8946c07 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:50.205956 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972d59e5-7ba0-4177-bec4-1762277e4cbf None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:50.209681 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93212f09-d7f8-4a5e-894f-18d2d8946c07 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:50.241783 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1bd05c-c3d0-4b9a-920b-87abf8c39e69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:50.242407 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dec7792-fd29-4622-8ec0-f9e3b720d35f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:50.242767 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1bd05c-c3d0-4b9a-920b-87abf8c39e69 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:50.242857 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dec7792-fd29-4622-8ec0-f9e3b720d35f None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:50.242952 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1bd05c-c3d0-4b9a-920b-87abf8c39e69 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:50.243032 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dec7792-fd29-4622-8ec0-f9e3b720d35f None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:50.243110 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1bd05c-c3d0-4b9a-920b-87abf8c39e69 None admin] PATH_INFO: `/v3/roles` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:50.243694 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dec7792-fd29-4622-8ec0-f9e3b720d35f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:50.244041 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1bd05c-c3d0-4b9a-920b-87abf8c39e69 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:03:50.244343 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1bd05c-c3d0-4b9a-920b-87abf8c39e69 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:50.244513 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dec7792-fd29-4622-8ec0-f9e3b720d35f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:50.245741 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1bd05c-c3d0-4b9a-920b-87abf8c39e69 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:50.245974 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dec7792-fd29-4622-8ec0-f9e3b720d35f None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:50.247530 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 144/306] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 01:03:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:03:50.248630 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e337ff82-10ee-4ded-a870-8ae549e6204b None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:50.248945 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e337ff82-10ee-4ded-a870-8ae549e6204b None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:50.249224 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e337ff82-10ee-4ded-a870-8ae549e6204b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:50.263426 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 163/307] 10.222.0.20 () {56 vars in 1101 bytes} [Wed Aug 31 01:03:50 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:03:50.263426 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1bd05c-c3d0-4b9a-920b-87abf8c39e69 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:50.266937 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-e337ff82-10ee-4ded-a870-8ae549e6204b None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:03:50.268126 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-e337ff82-10ee-4ded-a870-8ae549e6204b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:03:50.281537 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:50.308318 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:50.335384 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 145/308] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:03:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 01:03:50.336412 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e337ff82-10ee-4ded-a870-8ae549e6204b None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:50.375775 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1bd05c-c3d0-4b9a-920b-87abf8c39e69 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:50.412955 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f95f6cc-8eb2-47d5-b884-94a1e1da5bf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:50.413610 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f95f6cc-8eb2-47d5-b884-94a1e1da5bf6 None admin] REQUEST_METHOD: `PUT` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:50.413891 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f95f6cc-8eb2-47d5-b884-94a1e1da5bf6 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:50.414167 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f95f6cc-8eb2-47d5-b884-94a1e1da5bf6 None admin] PATH_INFO: `/v3/system/users/f0979995b04449ba816ce361cc47a953/roles/d0b9768f4d014766b9326387f2e11f34` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:50.415618 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f95f6cc-8eb2-47d5-b884-94a1e1da5bf6 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=f0979995b04449ba816ce361cc47a953, role_id=d0b9768f4d014766b9326387f2e11f34)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:50.417028 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f95f6cc-8eb2-47d5-b884-94a1e1da5bf6 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:50.442098 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e337ff82-10ee-4ded-a870-8ae549e6204b None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:50.476533 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eee9f56-d7b4-4265-9674-a59da082be01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:50.477184 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eee9f56-d7b4-4265-9674-a59da082be01 None admin] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:50.477458 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eee9f56-d7b4-4265-9674-a59da082be01 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:50.477756 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eee9f56-d7b4-4265-9674-a59da082be01 None admin] PATH_INFO: `/v3/groups` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:50.478381 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eee9f56-d7b4-4265-9674-a59da082be01 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:50.479807 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eee9f56-d7b4-4265-9674-a59da082be01 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:50.503158 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 164/309] 10.222.0.20 () {58 vars in 1334 bytes} [Wed Aug 31 01:03:50 2022] PUT /identity/v3/system/users/f0979995b04449ba816ce361cc47a953/roles/d0b9768f4d014766b9326387f2e11f34 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:03:50.504931 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f95f6cc-8eb2-47d5-b884-94a1e1da5bf6 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:50.585083 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 146/310] 10.222.0.20 () {60 vars in 1134 bytes} [Wed Aug 31 01:03:50 2022] POST /identity/v3/groups => generated 229 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:03:50.612584 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eee9f56-d7b4-4265-9674-a59da082be01 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:50.625382 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:50.639903 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f95f6cc-8eb2-47d5-b884-94a1e1da5bf6 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:50.652792 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:50.695267 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9280225a-e8d0-4cd1-9f53-392fe301293b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:50.696607 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9280225a-e8d0-4cd1-9f53-392fe301293b None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:50.697127 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9280225a-e8d0-4cd1-9f53-392fe301293b None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:50.697605 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9280225a-e8d0-4cd1-9f53-392fe301293b None admin] PATH_INFO: `/v3/users/df387b9f2fa6483eb82d2b0934a9a274` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:50.718106 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9280225a-e8d0-4cd1-9f53-392fe301293b None admin] RBAC: Authorizing `identity:get_user(user_id=df387b9f2fa6483eb82d2b0934a9a274)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:50.720855 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9280225a-e8d0-4cd1-9f53-392fe301293b None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:50.723942 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 165/311] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Aug 31 01:03:50 2022] GET /identity/v3/users/df387b9f2fa6483eb82d2b0934a9a274 => generated 293 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:03:50.732351 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9280225a-e8d0-4cd1-9f53-392fe301293b None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:50.741600 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eee9f56-d7b4-4265-9674-a59da082be01 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:50.783480 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab52f702-305f-4e66-bdd1-5ae6d7bbab92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:50.783823 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab52f702-305f-4e66-bdd1-5ae6d7bbab92 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:50.784046 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab52f702-305f-4e66-bdd1-5ae6d7bbab92 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:50.784315 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab52f702-305f-4e66-bdd1-5ae6d7bbab92 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:50.798936 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab52f702-305f-4e66-bdd1-5ae6d7bbab92 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 01:03:50.804593 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application [None req-ab52f702-305f-4e66-bdd1-5ae6d7bbab92 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 01:03:50.804593 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:03:50.804593 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:03:50.804593 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:03:50.804593 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:03:50.804593 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:03:50.804593 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:03:50.804593 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:03:50.804593 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:03:50.804593 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:03:50.804593 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:03:50.804593 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:03:50.804593 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 01:03:50.804593 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 01:03:50.804593 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 01:03:50.804593 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 01:03:50.805511 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 01:03:50.805511 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 01:03:50.805511 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 01:03:50.805511 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 01:03:50.805511 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:03:50.805511 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:03:50.805511 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:03:50.805511 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:03:50.805511 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:03:50.805511 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:03:50.805511 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:03:50.805511 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application with Lock( Aug 31 01:03:50.805511 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:03:50.805511 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:03:50.805511 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:03:50.805511 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:03:50.807662 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:03:50.807662 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:03:50.807662 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:03:50.807662 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:03:50.807662 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 01:03:50.807662 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 01:03:50.807662 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 01:03:50.807662 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 01:03:50.807662 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 01:03:50.807662 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 01:03:50.807662 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 01:03:50.807662 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Aug 31 01:03:50.807662 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 147/312] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Aug 31 01:03:50 2022] GET /identity/v3/roles/reader => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 01:03:50.816902 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab52f702-305f-4e66-bdd1-5ae6d7bbab92 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:50.863738 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9280225a-e8d0-4cd1-9f53-392fe301293b None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:50.895528 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:50.904954 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a17993c-f94f-41a0-9e32-2eecf8d3c647 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:50.905640 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a17993c-f94f-41a0-9e32-2eecf8d3c647 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:50.905934 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a17993c-f94f-41a0-9e32-2eecf8d3c647 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:50.906205 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a17993c-f94f-41a0-9e32-2eecf8d3c647 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:50.907793 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a17993c-f94f-41a0-9e32-2eecf8d3c647 None admin] RBAC: Adding query filter params (role.id=e1aef2df18ab425a8b78190b0fbfe8ba, user.id=df387b9f2fa6483eb82d2b0934a9a274, scope.system=all) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:03:50.907793 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a17993c-f94f-41a0-9e32-2eecf8d3c647 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:50.908669 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a17993c-f94f-41a0-9e32-2eecf8d3c647 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:50.914034 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 166/313] 10.222.0.20 () {56 vars in 1308 bytes} [Wed Aug 31 01:03:50 2022] GET /identity/v3/role_assignments?user.id=df387b9f2fa6483eb82d2b0934a9a274&scope.system=all&role.id=e1aef2df18ab425a8b78190b0fbfe8ba => generated 226 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 01:03:50.971589 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab52f702-305f-4e66-bdd1-5ae6d7bbab92 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:51.011065 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7379ec-95c4-4a01-887a-0abff59d1aa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:51.011691 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7379ec-95c4-4a01-887a-0abff59d1aa6 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:51.011972 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7379ec-95c4-4a01-887a-0abff59d1aa6 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:51.012246 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7379ec-95c4-4a01-887a-0abff59d1aa6 None admin] PATH_INFO: `/v3/roles` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:51.012966 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7379ec-95c4-4a01-887a-0abff59d1aa6 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:03:51.013276 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7379ec-95c4-4a01-887a-0abff59d1aa6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:51.014772 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7379ec-95c4-4a01-887a-0abff59d1aa6 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:51.024703 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 148/314] 10.222.0.20 () {56 vars in 1101 bytes} [Wed Aug 31 01:03:50 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:03:51.030463 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a17993c-f94f-41a0-9e32-2eecf8d3c647 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:51.064389 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:51.137107 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a17993c-f94f-41a0-9e32-2eecf8d3c647 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:51.172669 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6958bf-fb50-4931-9374-4a6e7ed07607 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:51.173448 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6958bf-fb50-4931-9374-4a6e7ed07607 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:51.173802 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6958bf-fb50-4931-9374-4a6e7ed07607 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:51.174147 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6958bf-fb50-4931-9374-4a6e7ed07607 None admin] PATH_INFO: `/v3/users/49f42887f51c497d8ae8ff3f7eb71e07` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:51.175357 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6958bf-fb50-4931-9374-4a6e7ed07607 None admin] RBAC: Authorizing `identity:get_user(user_id=49f42887f51c497d8ae8ff3f7eb71e07)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:51.176975 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6958bf-fb50-4931-9374-4a6e7ed07607 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:51.178971 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 167/315] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Aug 31 01:03:51 2022] GET /identity/v3/users/49f42887f51c497d8ae8ff3f7eb71e07 => generated 297 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:03:51.184588 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7379ec-95c4-4a01-887a-0abff59d1aa6 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:51.293142 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7379ec-95c4-4a01-887a-0abff59d1aa6 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:51.325073 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:51.328302 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729aae09-1942-4da8-a7d8-2e7a1d8a3ce2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:51.328952 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729aae09-1942-4da8-a7d8-2e7a1d8a3ce2 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:51.329305 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729aae09-1942-4da8-a7d8-2e7a1d8a3ce2 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:51.329587 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729aae09-1942-4da8-a7d8-2e7a1d8a3ce2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:51.400852 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729aae09-1942-4da8-a7d8-2e7a1d8a3ce2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:51.402327 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729aae09-1942-4da8-a7d8-2e7a1d8a3ce2 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:51.503190 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 149/316] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Aug 31 01:03:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 31 01:03:51.509600 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6958bf-fb50-4931-9374-4a6e7ed07607 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:51.612646 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:51.617093 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6958bf-fb50-4931-9374-4a6e7ed07607 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:51.654539 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95e9d3a-3ffb-419a-9a6d-8e03632ab7d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:51.655097 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95e9d3a-3ffb-419a-9a6d-8e03632ab7d5 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:51.655379 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95e9d3a-3ffb-419a-9a6d-8e03632ab7d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:51.655655 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95e9d3a-3ffb-419a-9a6d-8e03632ab7d5 None admin] PATH_INFO: `/v3/projects/b4ae9d03e35549cfbadf1d1001837bb6` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:51.656788 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95e9d3a-3ffb-419a-9a6d-8e03632ab7d5 None admin] RBAC: Authorizing `identity:get_project(project_id=b4ae9d03e35549cfbadf1d1001837bb6)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:51.658223 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95e9d3a-3ffb-419a-9a6d-8e03632ab7d5 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:51.660046 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 168/317] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Aug 31 01:03:51 2022] GET /identity/v3/projects/b4ae9d03e35549cfbadf1d1001837bb6 => generated 302 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:03:51.666566 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729aae09-1942-4da8-a7d8-2e7a1d8a3ce2 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:51.701642 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:51.775051 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729aae09-1942-4da8-a7d8-2e7a1d8a3ce2 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:51.812392 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cab76a1-fefe-4ce6-a60f-60721f424aea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:51.813042 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cab76a1-fefe-4ce6-a60f-60721f424aea None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:51.813322 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cab76a1-fefe-4ce6-a60f-60721f424aea None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:51.814211 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cab76a1-fefe-4ce6-a60f-60721f424aea None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:51.814472 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cab76a1-fefe-4ce6-a60f-60721f424aea None admin] RBAC: Adding query filter params (role.id=e1aef2df18ab425a8b78190b0fbfe8ba, scope.project.id=b4ae9d03e35549cfbadf1d1001837bb6, user.id=49f42887f51c497d8ae8ff3f7eb71e07) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:03:51.814651 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cab76a1-fefe-4ce6-a60f-60721f424aea None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:51.816062 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cab76a1-fefe-4ce6-a60f-60721f424aea None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:51.823663 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 150/318] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Aug 31 01:03:51 2022] GET /identity/v3/role_assignments?user.id=49f42887f51c497d8ae8ff3f7eb71e07&scope.project.id=b4ae9d03e35549cfbadf1d1001837bb6&role.id=e1aef2df18ab425a8b78190b0fbfe8ba => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:03:52.460366 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00746412-cded-4205-9bb7-ded6b7717865 None None] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:52.460670 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00746412-cded-4205-9bb7-ded6b7717865 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:52.460933 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00746412-cded-4205-9bb7-ded6b7717865 None None] PATH_INFO: `/` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:52.462832 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 169/319] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:03:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:03:52.467832 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72537de2-2ab8-4e35-9920-1a55f4bbd1c3 None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:52.468274 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72537de2-2ab8-4e35-9920-1a55f4bbd1c3 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:52.468431 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72537de2-2ab8-4e35-9920-1a55f4bbd1c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:52.486692 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-72537de2-2ab8-4e35-9920-1a55f4bbd1c3 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:03:52.487861 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-72537de2-2ab8-4e35-9920-1a55f4bbd1c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:03:52.553953 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 151/320] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:03:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 01:03:52.681218 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b820c37-d1c1-4f79-8935-d325ca84ad8f None None] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:52.681721 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b820c37-d1c1-4f79-8935-d325ca84ad8f None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:52.681794 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b820c37-d1c1-4f79-8935-d325ca84ad8f None None] PATH_INFO: `/` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:52.683269 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 170/321] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:03:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:03:52.688135 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084d4199-c375-426e-bb50-1cd39c363ee5 None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:52.688432 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084d4199-c375-426e-bb50-1cd39c363ee5 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:52.688706 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084d4199-c375-426e-bb50-1cd39c363ee5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:52.706871 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-084d4199-c375-426e-bb50-1cd39c363ee5 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:03:52.708049 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-084d4199-c375-426e-bb50-1cd39c363ee5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:03:52.726762 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:52.776576 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 152/322] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:03:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 01:03:52.818277 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b820c37-d1c1-4f79-8935-d325ca84ad8f None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:52.915716 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f8b91a-2bbe-4027-9d79-eb2e4e604bb1 None None] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:52.916111 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f8b91a-2bbe-4027-9d79-eb2e4e604bb1 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:52.916396 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f8b91a-2bbe-4027-9d79-eb2e4e604bb1 None None] PATH_INFO: `/` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:52.918000 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 153/323] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:03:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:03:52.922716 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c580ee-5ac3-454e-9e15-83cfcdd960dc None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:52.923005 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c580ee-5ac3-454e-9e15-83cfcdd960dc None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:52.923285 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c580ee-5ac3-454e-9e15-83cfcdd960dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:52.925255 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b820c37-d1c1-4f79-8935-d325ca84ad8f None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:52.941130 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-34c580ee-5ac3-454e-9e15-83cfcdd960dc None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:03:52.942310 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-34c580ee-5ac3-454e-9e15-83cfcdd960dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:03:52.960122 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2c74f0-777d-4d2a-9c60-f1a48ac23584 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:52.960779 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2c74f0-777d-4d2a-9c60-f1a48ac23584 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:52.961056 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2c74f0-777d-4d2a-9c60-f1a48ac23584 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:52.961324 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2c74f0-777d-4d2a-9c60-f1a48ac23584 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:52.971392 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2c74f0-777d-4d2a-9c60-f1a48ac23584 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 01:03:52.976205 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application [None req-6d2c74f0-777d-4d2a-9c60-f1a48ac23584 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 01:03:52.976205 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:03:52.976205 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:03:52.976205 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:03:52.976205 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:03:52.976205 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:03:52.976205 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:03:52.976205 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:03:52.976205 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:03:52.976205 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:03:52.976205 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:03:52.976205 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:03:52.976205 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 01:03:52.976205 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 01:03:52.976205 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 01:03:52.976205 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 01:03:52.977364 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 01:03:52.977364 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 01:03:52.977364 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 01:03:52.977364 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 01:03:52.977364 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:03:52.977364 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:03:52.977364 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:03:52.977364 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:03:52.977364 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:03:52.977364 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:03:52.977364 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:03:52.977364 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application with Lock( Aug 31 01:03:52.977364 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:03:52.977364 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:03:52.977364 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:03:52.977364 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:03:52.978879 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:03:52.978879 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:03:52.978879 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:03:52.978879 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:03:52.978879 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 01:03:52.978879 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 01:03:52.978879 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 01:03:52.978879 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 01:03:52.978879 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 01:03:52.978879 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 01:03:52.978879 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 01:03:52.978879 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Aug 31 01:03:52.978879 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 171/324] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Aug 31 01:03:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:03:52.985426 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2c74f0-777d-4d2a-9c60-f1a48ac23584 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:53.014021 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:53.019289 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 154/325] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:03:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 01:03:53.042642 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c580ee-5ac3-454e-9e15-83cfcdd960dc None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:53.099713 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2c74f0-777d-4d2a-9c60-f1a48ac23584 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:53.129582 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:53.136140 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f977ef3-85c0-4e75-b674-8cf09ff55e9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:53.136862 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f977ef3-85c0-4e75-b674-8cf09ff55e9c None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:53.137052 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f977ef3-85c0-4e75-b674-8cf09ff55e9c None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:53.137330 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f977ef3-85c0-4e75-b674-8cf09ff55e9c None admin] PATH_INFO: `/v3/roles` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:53.137988 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f977ef3-85c0-4e75-b674-8cf09ff55e9c None admin] RBAC: Adding query filter params (name=member) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:03:53.138293 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f977ef3-85c0-4e75-b674-8cf09ff55e9c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:53.142806 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f977ef3-85c0-4e75-b674-8cf09ff55e9c None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:53.144987 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c580ee-5ac3-454e-9e15-83cfcdd960dc None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:53.147942 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 172/326] 10.222.0.20 () {56 vars in 1101 bytes} [Wed Aug 31 01:03:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 01:03:53.153200 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f977ef3-85c0-4e75-b674-8cf09ff55e9c None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:53.183200 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32121b6-672b-4128-9ca2-3ac0387f1160 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:53.183831 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32121b6-672b-4128-9ca2-3ac0387f1160 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:53.184119 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32121b6-672b-4128-9ca2-3ac0387f1160 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:53.184394 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32121b6-672b-4128-9ca2-3ac0387f1160 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:53.185537 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32121b6-672b-4128-9ca2-3ac0387f1160 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:53.186982 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32121b6-672b-4128-9ca2-3ac0387f1160 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:53.188542 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 155/327] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 01:03:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 01:03:53.193900 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32121b6-672b-4128-9ca2-3ac0387f1160 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:53.247034 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:53.259600 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f977ef3-85c0-4e75-b674-8cf09ff55e9c None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:53.297842 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32121b6-672b-4128-9ca2-3ac0387f1160 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:53.299284 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a51c4e3-dbc7-4e99-8b1b-c7f8f1865245 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:53.299915 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a51c4e3-dbc7-4e99-8b1b-c7f8f1865245 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:53.300201 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a51c4e3-dbc7-4e99-8b1b-c7f8f1865245 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:53.300476 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a51c4e3-dbc7-4e99-8b1b-c7f8f1865245 None admin] PATH_INFO: `/v3/users/f0979995b04449ba816ce361cc47a953` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:53.301729 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a51c4e3-dbc7-4e99-8b1b-c7f8f1865245 None admin] RBAC: Authorizing `identity:get_user(user_id=f0979995b04449ba816ce361cc47a953)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:53.303182 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a51c4e3-dbc7-4e99-8b1b-c7f8f1865245 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:53.304774 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 173/328] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Aug 31 01:03:53 2022] GET /identity/v3/users/f0979995b04449ba816ce361cc47a953 => generated 293 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 01:03:53.305550 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a51c4e3-dbc7-4e99-8b1b-c7f8f1865245 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:53.331550 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf90461-242e-497a-858b-ae6aed6bc074 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:53.332151 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf90461-242e-497a-858b-ae6aed6bc074 None admin] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:53.332433 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf90461-242e-497a-858b-ae6aed6bc074 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:53.332713 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf90461-242e-497a-858b-ae6aed6bc074 None admin] PATH_INFO: `/v3/groups` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:53.333373 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf90461-242e-497a-858b-ae6aed6bc074 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:53.334830 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf90461-242e-497a-858b-ae6aed6bc074 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:53.402870 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a51c4e3-dbc7-4e99-8b1b-c7f8f1865245 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:53.406357 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 156/329] 10.222.0.20 () {60 vars in 1134 bytes} [Wed Aug 31 01:03:53 2022] POST /identity/v3/groups => generated 226 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 01:03:53.412247 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf90461-242e-497a-858b-ae6aed6bc074 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:53.424988 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:53.446072 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39be3b3d-2c8d-47de-87c8-ab13a1c105a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:53.446771 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39be3b3d-2c8d-47de-87c8-ab13a1c105a2 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:53.447048 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39be3b3d-2c8d-47de-87c8-ab13a1c105a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:53.447326 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39be3b3d-2c8d-47de-87c8-ab13a1c105a2 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:53.459977 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39be3b3d-2c8d-47de-87c8-ab13a1c105a2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 01:03:53.466226 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application [None req-39be3b3d-2c8d-47de-87c8-ab13a1c105a2 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 01:03:53.466226 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:03:53.466226 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:03:53.466226 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:03:53.466226 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:03:53.466226 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:03:53.466226 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:03:53.466226 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:03:53.466226 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:03:53.466226 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:03:53.466226 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:03:53.466226 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:03:53.466226 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 01:03:53.466226 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 01:03:53.466226 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 01:03:53.466226 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 01:03:53.467180 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 01:03:53.467180 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 01:03:53.467180 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 01:03:53.467180 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 01:03:53.467180 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:03:53.467180 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:03:53.467180 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:03:53.467180 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:03:53.467180 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:03:53.467180 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:03:53.467180 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:03:53.467180 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application with Lock( Aug 31 01:03:53.467180 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:03:53.467180 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:03:53.467180 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:03:53.467180 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:03:53.467978 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:03:53.467978 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:03:53.467978 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:03:53.467978 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:03:53.467978 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 01:03:53.467978 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 01:03:53.467978 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 01:03:53.467978 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 01:03:53.467978 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 01:03:53.467978 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 01:03:53.467978 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 01:03:53.467978 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Aug 31 01:03:53.467978 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 174/330] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Aug 31 01:03:53 2022] GET /identity/v3/roles/reader => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:03:53.475136 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39be3b3d-2c8d-47de-87c8-ab13a1c105a2 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:53.499386 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:53.524927 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf90461-242e-497a-858b-ae6aed6bc074 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:53.566851 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cba7b10-9f69-40e8-a2b8-5f48b9213072 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:53.567486 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cba7b10-9f69-40e8-a2b8-5f48b9213072 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:53.567778 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cba7b10-9f69-40e8-a2b8-5f48b9213072 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:53.568060 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cba7b10-9f69-40e8-a2b8-5f48b9213072 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:53.568847 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cba7b10-9f69-40e8-a2b8-5f48b9213072 None admin] RBAC: Adding query filter params (role.id=d0b9768f4d014766b9326387f2e11f34, user.id=f0979995b04449ba816ce361cc47a953, scope.system=all) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:03:53.569166 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cba7b10-9f69-40e8-a2b8-5f48b9213072 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:53.570613 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cba7b10-9f69-40e8-a2b8-5f48b9213072 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:53.583566 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 157/331] 10.222.0.20 () {56 vars in 1308 bytes} [Wed Aug 31 01:03:53 2022] GET /identity/v3/role_assignments?user.id=f0979995b04449ba816ce361cc47a953&scope.system=all&role.id=d0b9768f4d014766b9326387f2e11f34 => generated 510 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:03:53.596276 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39be3b3d-2c8d-47de-87c8-ab13a1c105a2 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:53.637568 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fabe0b-bd97-4991-8c79-90d9c65ac868 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:53.638211 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fabe0b-bd97-4991-8c79-90d9c65ac868 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:53.638488 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fabe0b-bd97-4991-8c79-90d9c65ac868 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:53.638774 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fabe0b-bd97-4991-8c79-90d9c65ac868 None admin] PATH_INFO: `/v3/roles` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:53.639463 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fabe0b-bd97-4991-8c79-90d9c65ac868 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:03:53.639779 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fabe0b-bd97-4991-8c79-90d9c65ac868 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:53.641238 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fabe0b-bd97-4991-8c79-90d9c65ac868 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:53.651537 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 175/332] 10.222.0.20 () {56 vars in 1101 bytes} [Wed Aug 31 01:03:53 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 01:03:53.657320 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cba7b10-9f69-40e8-a2b8-5f48b9213072 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:53.738736 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:53.771936 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cba7b10-9f69-40e8-a2b8-5f48b9213072 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:53.812944 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7370d7b6-8d15-47ba-bc45-ed98b73aa122 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:53.813623 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7370d7b6-8d15-47ba-bc45-ed98b73aa122 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:53.813906 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7370d7b6-8d15-47ba-bc45-ed98b73aa122 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:53.814183 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7370d7b6-8d15-47ba-bc45-ed98b73aa122 None admin] PATH_INFO: `/v3/users/df387b9f2fa6483eb82d2b0934a9a274` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:53.815341 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7370d7b6-8d15-47ba-bc45-ed98b73aa122 None admin] RBAC: Authorizing `identity:get_user(user_id=df387b9f2fa6483eb82d2b0934a9a274)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:53.816872 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7370d7b6-8d15-47ba-bc45-ed98b73aa122 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:53.818769 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 158/333] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Aug 31 01:03:53 2022] GET /identity/v3/users/df387b9f2fa6483eb82d2b0934a9a274 => generated 293 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:03:53.825252 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fabe0b-bd97-4991-8c79-90d9c65ac868 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:53.921686 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:53.942771 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fabe0b-bd97-4991-8c79-90d9c65ac868 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:53.982331 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162e0d7b-0ef1-4d79-a17b-356f3eb03e5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:53.982986 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162e0d7b-0ef1-4d79-a17b-356f3eb03e5f None admin] REQUEST_METHOD: `PUT` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:53.983267 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162e0d7b-0ef1-4d79-a17b-356f3eb03e5f None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:53.983541 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162e0d7b-0ef1-4d79-a17b-356f3eb03e5f None admin] PATH_INFO: `/v3/system/users/df387b9f2fa6483eb82d2b0934a9a274/roles/e1aef2df18ab425a8b78190b0fbfe8ba` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:53.985001 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162e0d7b-0ef1-4d79-a17b-356f3eb03e5f None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=df387b9f2fa6483eb82d2b0934a9a274, role_id=e1aef2df18ab425a8b78190b0fbfe8ba)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:53.986384 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162e0d7b-0ef1-4d79-a17b-356f3eb03e5f None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:54.048324 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 176/334] 10.222.0.20 () {58 vars in 1334 bytes} [Wed Aug 31 01:03:53 2022] PUT /identity/v3/system/users/df387b9f2fa6483eb82d2b0934a9a274/roles/e1aef2df18ab425a8b78190b0fbfe8ba => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:03:56.049157 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478dbcd1-5d95-4ae6-8b0a-6dfe83e8a487 None None] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:56.049731 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478dbcd1-5d95-4ae6-8b0a-6dfe83e8a487 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:56.049731 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478dbcd1-5d95-4ae6-8b0a-6dfe83e8a487 None None] PATH_INFO: `/` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:56.051019 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 159/335] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:03:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:03:56.056699 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa00fd0-8d2c-4b98-8a31-17d6819d8b7d None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:56.057168 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa00fd0-8d2c-4b98-8a31-17d6819d8b7d None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:56.057258 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa00fd0-8d2c-4b98-8a31-17d6819d8b7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:56.079272 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-0fa00fd0-8d2c-4b98-8a31-17d6819d8b7d None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:03:56.080459 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-0fa00fd0-8d2c-4b98-8a31-17d6819d8b7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:03:56.154958 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7847990e-45bf-499a-afd2-e6e2efc4744d None None] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:56.155255 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7847990e-45bf-499a-afd2-e6e2efc4744d None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:56.155620 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7847990e-45bf-499a-afd2-e6e2efc4744d None None] PATH_INFO: `/` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:56.157696 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 160/336] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:03:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 01:03:56.169351 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 177/337] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:03:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 01:03:56.174348 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3c0261-2b0e-48d7-b1d6-d97d66f7cdd0 None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:56.174630 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3c0261-2b0e-48d7-b1d6-d97d66f7cdd0 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:56.174908 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3c0261-2b0e-48d7-b1d6-d97d66f7cdd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:56.209777 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-0a3c0261-2b0e-48d7-b1d6-d97d66f7cdd0 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:03:56.210975 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-0a3c0261-2b0e-48d7-b1d6-d97d66f7cdd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:03:56.306873 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 161/338] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:03:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 01:03:56.415380 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84632d0-53c1-4e8b-8ea2-2fb6cb63d851 None None] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:56.415681 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84632d0-53c1-4e8b-8ea2-2fb6cb63d851 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:56.415854 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84632d0-53c1-4e8b-8ea2-2fb6cb63d851 None None] PATH_INFO: `/` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:56.417513 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 178/339] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:03:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:03:56.424520 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b921d04e-de40-496e-a2d6-1c512a07b2b6 None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:56.424833 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b921d04e-de40-496e-a2d6-1c512a07b2b6 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:56.425113 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b921d04e-de40-496e-a2d6-1c512a07b2b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:56.453691 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-b921d04e-de40-496e-a2d6-1c512a07b2b6 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:03:56.454890 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-b921d04e-de40-496e-a2d6-1c512a07b2b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:03:56.517136 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74b1d58-773b-4901-8033-90a9cba3b37a None None] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:56.517439 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74b1d58-773b-4901-8033-90a9cba3b37a None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:56.517699 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74b1d58-773b-4901-8033-90a9cba3b37a None None] PATH_INFO: `/` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:56.519308 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 179/340] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:03:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:03:56.528178 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:56.530012 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff17ed30-6ecc-4637-b6d5-67de55c62998 None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:56.530355 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff17ed30-6ecc-4637-b6d5-67de55c62998 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:56.530676 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff17ed30-6ecc-4637-b6d5-67de55c62998 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:56.534007 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 162/341] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:03:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 01:03:56.549456 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-ff17ed30-6ecc-4637-b6d5-67de55c62998 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:03:56.550674 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-ff17ed30-6ecc-4637-b6d5-67de55c62998 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:03:56.620120 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 180/342] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:03:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 01:03:56.661934 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b921d04e-de40-496e-a2d6-1c512a07b2b6 None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:56.771321 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b921d04e-de40-496e-a2d6-1c512a07b2b6 None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:56.809512 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e30ecb-8001-4c59-a56b-bec81e925f80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:56.810131 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e30ecb-8001-4c59-a56b-bec81e925f80 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:56.810409 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e30ecb-8001-4c59-a56b-bec81e925f80 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:56.810681 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e30ecb-8001-4c59-a56b-bec81e925f80 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:56.821021 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e30ecb-8001-4c59-a56b-bec81e925f80 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 01:03:56.828366 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application [None req-c1e30ecb-8001-4c59-a56b-bec81e925f80 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 01:03:56.828366 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:03:56.828366 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:03:56.828366 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:03:56.828366 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:03:56.828366 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:03:56.828366 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:03:56.828366 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:03:56.828366 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:03:56.828366 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:03:56.828366 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:03:56.828366 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:03:56.828366 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 01:03:56.828366 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 01:03:56.828366 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 01:03:56.828366 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 01:03:56.829418 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 01:03:56.829418 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 01:03:56.829418 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 01:03:56.829418 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 01:03:56.829418 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:03:56.829418 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:03:56.829418 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:03:56.829418 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:03:56.829418 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:03:56.829418 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:03:56.829418 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:03:56.829418 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application with Lock( Aug 31 01:03:56.829418 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:03:56.829418 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:03:56.829418 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:03:56.829418 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:03:56.830227 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:03:56.830227 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:03:56.830227 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:03:56.830227 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:03:56.830227 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 01:03:56.830227 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 01:03:56.830227 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 01:03:56.830227 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 01:03:56.830227 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 01:03:56.830227 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 01:03:56.830227 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 01:03:56.830227 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Aug 31 01:03:56.830227 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 163/343] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Aug 31 01:03:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:03:56.838051 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff17ed30-6ecc-4637-b6d5-67de55c62998 None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:56.840671 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:56.870522 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e30ecb-8001-4c59-a56b-bec81e925f80 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:57.009172 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff17ed30-6ecc-4637-b6d5-67de55c62998 None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:57.027081 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:57.050884 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a1642a-06eb-41f8-a576-35110081929a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:57.051506 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a1642a-06eb-41f8-a576-35110081929a None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:57.051793 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a1642a-06eb-41f8-a576-35110081929a None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:57.052042 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a1642a-06eb-41f8-a576-35110081929a None admin] PATH_INFO: `/v3/roles` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:57.052705 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a1642a-06eb-41f8-a576-35110081929a None admin] RBAC: Adding query filter params (name=member) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:03:57.053035 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a1642a-06eb-41f8-a576-35110081929a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:57.055114 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a1642a-06eb-41f8-a576-35110081929a None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:57.065353 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 181/344] 10.222.0.20 () {56 vars in 1101 bytes} [Wed Aug 31 01:03:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:03:57.066209 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a1642a-06eb-41f8-a576-35110081929a None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:57.125908 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e30ecb-8001-4c59-a56b-bec81e925f80 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:57.135431 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:57.168249 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610a4cd1-c229-4c58-8283-d89010f26a2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:57.169099 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610a4cd1-c229-4c58-8283-d89010f26a2b None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:57.169488 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610a4cd1-c229-4c58-8283-d89010f26a2b None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:57.169870 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610a4cd1-c229-4c58-8283-d89010f26a2b None admin] PATH_INFO: `/v3/roles/member` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:57.175017 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a1642a-06eb-41f8-a576-35110081929a None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:57.184890 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610a4cd1-c229-4c58-8283-d89010f26a2b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 01:03:57.190853 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application [None req-610a4cd1-c229-4c58-8283-d89010f26a2b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 01:03:57.190853 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:03:57.190853 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:03:57.190853 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:03:57.190853 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:03:57.190853 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:03:57.190853 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:03:57.190853 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:03:57.190853 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:03:57.190853 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:03:57.190853 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:03:57.190853 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:03:57.190853 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 01:03:57.190853 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 01:03:57.190853 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 01:03:57.190853 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 01:03:57.191940 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 01:03:57.191940 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 01:03:57.191940 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 01:03:57.191940 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 01:03:57.191940 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:03:57.191940 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:03:57.191940 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:03:57.191940 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:03:57.191940 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:03:57.191940 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:03:57.191940 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:03:57.191940 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application with Lock( Aug 31 01:03:57.191940 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:03:57.191940 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:03:57.191940 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:03:57.191940 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:03:57.192925 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:03:57.192925 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:03:57.192925 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:03:57.192925 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:03:57.192925 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 01:03:57.192925 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 01:03:57.192925 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 01:03:57.192925 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 01:03:57.192925 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 01:03:57.192925 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 01:03:57.192925 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 01:03:57.192925 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Aug 31 01:03:57.194909 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 164/345] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Aug 31 01:03:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 325 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:03:57.195836 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610a4cd1-c229-4c58-8283-d89010f26a2b None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:57.214940 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49989d8-6ae2-4bbd-9111-ba607ffcca75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:57.215579 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49989d8-6ae2-4bbd-9111-ba607ffcca75 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:57.215863 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49989d8-6ae2-4bbd-9111-ba607ffcca75 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:57.216137 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49989d8-6ae2-4bbd-9111-ba607ffcca75 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:57.227967 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49989d8-6ae2-4bbd-9111-ba607ffcca75 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 01:03:57.235958 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application [None req-d49989d8-6ae2-4bbd-9111-ba607ffcca75 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 01:03:57.235958 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:03:57.235958 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:03:57.235958 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:03:57.235958 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:03:57.235958 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:03:57.235958 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:03:57.235958 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:03:57.235958 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:03:57.235958 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:03:57.235958 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:03:57.235958 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:03:57.235958 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 01:03:57.235958 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 01:03:57.235958 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 01:03:57.235958 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 01:03:57.236909 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 01:03:57.236909 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 01:03:57.236909 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 01:03:57.236909 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 01:03:57.236909 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:03:57.236909 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:03:57.236909 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:03:57.236909 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:03:57.236909 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:03:57.236909 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:03:57.236909 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:03:57.236909 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application with Lock( Aug 31 01:03:57.236909 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:03:57.236909 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:03:57.236909 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:03:57.236909 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:03:57.237806 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:03:57.237806 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:03:57.237806 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:03:57.237806 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:03:57.237806 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 01:03:57.237806 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 01:03:57.237806 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 01:03:57.237806 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 01:03:57.237806 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 01:03:57.237806 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 01:03:57.237806 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 01:03:57.237806 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Aug 31 01:03:57.239120 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 182/346] 10.222.0.20 () {56 vars in 1114 bytes} [Wed Aug 31 01:03:57 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:03:57.239120 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49989d8-6ae2-4bbd-9111-ba607ffcca75 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:57.333884 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610a4cd1-c229-4c58-8283-d89010f26a2b None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:57.357654 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:57.373470 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49989d8-6ae2-4bbd-9111-ba607ffcca75 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:57.376596 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a5315d-f262-42d7-b6f1-e62d68f2a47e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:57.377274 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a5315d-f262-42d7-b6f1-e62d68f2a47e None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:57.377595 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a5315d-f262-42d7-b6f1-e62d68f2a47e None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:57.377839 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a5315d-f262-42d7-b6f1-e62d68f2a47e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:57.402477 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:57.417567 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1ce770-e74c-4ed0-b239-3c9ce14c9587 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:57.418590 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1ce770-e74c-4ed0-b239-3c9ce14c9587 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:57.419099 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1ce770-e74c-4ed0-b239-3c9ce14c9587 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:57.419605 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1ce770-e74c-4ed0-b239-3c9ce14c9587 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:57.434837 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1ce770-e74c-4ed0-b239-3c9ce14c9587 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 01:03:57.440250 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application [None req-7f1ce770-e74c-4ed0-b239-3c9ce14c9587 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 01:03:57.440250 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:03:57.440250 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:03:57.440250 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:03:57.440250 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:03:57.440250 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:03:57.440250 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:03:57.440250 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:03:57.440250 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:03:57.440250 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:03:57.440250 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:03:57.440250 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:03:57.440250 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 01:03:57.440250 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 01:03:57.440250 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 01:03:57.440250 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 01:03:57.441766 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 01:03:57.441766 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 01:03:57.441766 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 01:03:57.441766 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 01:03:57.441766 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:03:57.441766 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:03:57.441766 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:03:57.441766 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:03:57.441766 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:03:57.441766 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:03:57.441766 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:03:57.441766 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application with Lock( Aug 31 01:03:57.441766 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:03:57.441766 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:03:57.441766 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:03:57.441766 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:03:57.442692 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:03:57.442692 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:03:57.442692 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:03:57.442692 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:03:57.442692 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 01:03:57.442692 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 01:03:57.442692 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 01:03:57.442692 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 01:03:57.442692 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 01:03:57.442692 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 01:03:57.442692 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 01:03:57.442692 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Aug 31 01:03:57.444996 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 183/347] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Aug 31 01:03:57 2022] GET /identity/v3/roles/reader => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:03:57.447998 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1ce770-e74c-4ed0-b239-3c9ce14c9587 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:57.471829 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a5315d-f262-42d7-b6f1-e62d68f2a47e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:57.473460 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a5315d-f262-42d7-b6f1-e62d68f2a47e None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:57.581200 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1ce770-e74c-4ed0-b239-3c9ce14c9587 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:57.611291 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 165/348] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Aug 31 01:03:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 01:03:57.612022 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a5315d-f262-42d7-b6f1-e62d68f2a47e None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:57.625309 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee83f95-1e65-4abb-afed-262c7ce9156a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:57.626454 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee83f95-1e65-4abb-afed-262c7ce9156a None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:57.626919 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee83f95-1e65-4abb-afed-262c7ce9156a None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:57.627193 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee83f95-1e65-4abb-afed-262c7ce9156a None admin] PATH_INFO: `/v3/roles` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:57.628001 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee83f95-1e65-4abb-afed-262c7ce9156a None admin] RBAC: Adding query filter params (name=member) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:03:57.628308 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee83f95-1e65-4abb-afed-262c7ce9156a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:57.630201 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee83f95-1e65-4abb-afed-262c7ce9156a None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:57.634465 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:57.639909 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 184/349] 10.222.0.20 () {56 vars in 1101 bytes} [Wed Aug 31 01:03:57 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:03:57.641487 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e41a3a-fcef-4491-b97b-342f4da16c23 None None] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:57.642202 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e41a3a-fcef-4491-b97b-342f4da16c23 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:57.642693 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e41a3a-fcef-4491-b97b-342f4da16c23 None None] PATH_INFO: `/` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:57.644107 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 185/350] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:03:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:03:57.644998 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e41a3a-fcef-4491-b97b-342f4da16c23 None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:57.704798 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:57.749771 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a5315d-f262-42d7-b6f1-e62d68f2a47e None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:57.770371 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e41a3a-fcef-4491-b97b-342f4da16c23 None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:57.798412 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f283b73a-9619-4ca4-b0f5-7ac5b919af85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:57.799099 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f283b73a-9619-4ca4-b0f5-7ac5b919af85 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:57.799323 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f283b73a-9619-4ca4-b0f5-7ac5b919af85 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:57.799602 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f283b73a-9619-4ca4-b0f5-7ac5b919af85 None admin] PATH_INFO: `/v3/roles` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:57.800277 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f283b73a-9619-4ca4-b0f5-7ac5b919af85 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:03:57.800583 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f283b73a-9619-4ca4-b0f5-7ac5b919af85 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:57.802009 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f283b73a-9619-4ca4-b0f5-7ac5b919af85 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:57.812100 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 166/351] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Aug 31 01:03:57 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:03:57.813190 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f283b73a-9619-4ca4-b0f5-7ac5b919af85 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:57.816419 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d43a76-de98-4ed9-bef4-79437f5d9c4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:57.817530 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d43a76-de98-4ed9-bef4-79437f5d9c4c None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:57.817644 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d43a76-de98-4ed9-bef4-79437f5d9c4c None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:57.817937 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d43a76-de98-4ed9-bef4-79437f5d9c4c None admin] PATH_INFO: `/v3/roles` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:57.818896 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d43a76-de98-4ed9-bef4-79437f5d9c4c None admin] RBAC: Adding query filter params (name=reader) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:03:57.819208 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d43a76-de98-4ed9-bef4-79437f5d9c4c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:57.820596 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d43a76-de98-4ed9-bef4-79437f5d9c4c None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:57.832104 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 186/352] 10.222.0.20 () {56 vars in 1101 bytes} [Wed Aug 31 01:03:57 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:03:57.832987 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d43a76-de98-4ed9-bef4-79437f5d9c4c None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:57.931467 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:57.952453 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f283b73a-9619-4ca4-b0f5-7ac5b919af85 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:57.978899 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d43a76-de98-4ed9-bef4-79437f5d9c4c None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:57.990562 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:57.995439 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd2703a-88c1-41ad-b3dd-e0316af04e1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:57.996086 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd2703a-88c1-41ad-b3dd-e0316af04e1e None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:57.996363 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd2703a-88c1-41ad-b3dd-e0316af04e1e None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:57.996638 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd2703a-88c1-41ad-b3dd-e0316af04e1e None admin] PATH_INFO: `/v3/projects/b4ae9d03e35549cfbadf1d1001837bb6` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:57.997792 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd2703a-88c1-41ad-b3dd-e0316af04e1e None admin] RBAC: Authorizing `identity:get_project(project_id=b4ae9d03e35549cfbadf1d1001837bb6)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:57.999227 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd2703a-88c1-41ad-b3dd-e0316af04e1e None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:58.001030 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 167/353] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Aug 31 01:03:57 2022] GET /identity/v3/projects/b4ae9d03e35549cfbadf1d1001837bb6 => generated 302 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:03:58.002052 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0baedf9-6d63-4d80-a495-8e549961c470 None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:58.002334 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0baedf9-6d63-4d80-a495-8e549961c470 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:58.002607 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0baedf9-6d63-4d80-a495-8e549961c470 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:58.023524 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ca08a7-f800-4bfc-9f5b-8717599316e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:58.024290 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ca08a7-f800-4bfc-9f5b-8717599316e4 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:58.024421 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ca08a7-f800-4bfc-9f5b-8717599316e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:58.024680 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ca08a7-f800-4bfc-9f5b-8717599316e4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:58.025035 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-f0baedf9-6d63-4d80-a495-8e549961c470 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:03:58.026251 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-f0baedf9-6d63-4d80-a495-8e549961c470 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:03:58.100904 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 168/354] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:03:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 01:03:58.101610 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0baedf9-6d63-4d80-a495-8e549961c470 None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:58.110423 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ca08a7-f800-4bfc-9f5b-8717599316e4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:58.111891 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ca08a7-f800-4bfc-9f5b-8717599316e4 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:58.197770 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:58.232215 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0baedf9-6d63-4d80-a495-8e549961c470 None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:58.245882 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 187/355] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Aug 31 01:03:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 01:03:58.245882 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ca08a7-f800-4bfc-9f5b-8717599316e4 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:58.259586 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:58.288646 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f812770e-3749-46b8-93ea-507d4ac96804 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:58.289536 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f812770e-3749-46b8-93ea-507d4ac96804 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:58.290490 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f812770e-3749-46b8-93ea-507d4ac96804 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:58.290922 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f812770e-3749-46b8-93ea-507d4ac96804 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:58.371329 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f812770e-3749-46b8-93ea-507d4ac96804 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:58.373095 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f812770e-3749-46b8-93ea-507d4ac96804 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:58.376653 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ca08a7-f800-4bfc-9f5b-8717599316e4 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:58.415575 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f87ee1e-31e4-4450-9486-a252dad5b11a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:58.416212 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f87ee1e-31e4-4450-9486-a252dad5b11a None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:58.416502 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f87ee1e-31e4-4450-9486-a252dad5b11a None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:58.416829 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f87ee1e-31e4-4450-9486-a252dad5b11a None admin] PATH_INFO: `/v3/users/df387b9f2fa6483eb82d2b0934a9a274` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:58.418146 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f87ee1e-31e4-4450-9486-a252dad5b11a None admin] RBAC: Authorizing `identity:get_user(user_id=df387b9f2fa6483eb82d2b0934a9a274)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:58.419529 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f87ee1e-31e4-4450-9486-a252dad5b11a None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:58.421872 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 188/356] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Aug 31 01:03:58 2022] GET /identity/v3/users/df387b9f2fa6483eb82d2b0934a9a274 => generated 293 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:03:58.423835 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9eb9bdf-b9a2-44e9-9e17-22189ab298cc None None] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:58.423835 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9eb9bdf-b9a2-44e9-9e17-22189ab298cc None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:58.424028 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9eb9bdf-b9a2-44e9-9e17-22189ab298cc None None] PATH_INFO: `/` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:58.428923 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 189/357] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:03:58 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:03:58.429917 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9eb9bdf-b9a2-44e9-9e17-22189ab298cc None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:58.471974 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:58.495471 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 169/358] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Aug 31 01:03:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 01:03:58.498109 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f812770e-3749-46b8-93ea-507d4ac96804 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:58.532248 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:58.562557 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9eb9bdf-b9a2-44e9-9e17-22189ab298cc None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:58.608850 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4364cee4-1756-429d-a8ef-03a4f1b644f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:58.609490 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4364cee4-1756-429d-a8ef-03a4f1b644f9 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:58.609772 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4364cee4-1756-429d-a8ef-03a4f1b644f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:58.610042 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4364cee4-1756-429d-a8ef-03a4f1b644f9 None admin] PATH_INFO: `/v3/groups/73043087b04d4917b9fdcbd262ef56e0` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:58.610889 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f812770e-3749-46b8-93ea-507d4ac96804 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:58.617109 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4364cee4-1756-429d-a8ef-03a4f1b644f9 None admin] RBAC: Authorizing `identity:get_group(group_id=73043087b04d4917b9fdcbd262ef56e0)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:58.618607 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4364cee4-1756-429d-a8ef-03a4f1b644f9 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:58.620412 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 190/359] 10.222.0.20 () {56 vars in 1180 bytes} [Wed Aug 31 01:03:58 2022] GET /identity/v3/groups/73043087b04d4917b9fdcbd262ef56e0 => generated 226 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:03:58.621040 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4364cee4-1756-429d-a8ef-03a4f1b644f9 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:58.651471 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51ad8f7-4e61-445f-b0fd-101925dcd9d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:58.652443 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51ad8f7-4e61-445f-b0fd-101925dcd9d3 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:58.652752 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51ad8f7-4e61-445f-b0fd-101925dcd9d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:58.653021 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51ad8f7-4e61-445f-b0fd-101925dcd9d3 None admin] PATH_INFO: `/v3/projects/82238b119c5840de9fc817856cec36fb` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:58.654164 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51ad8f7-4e61-445f-b0fd-101925dcd9d3 None admin] RBAC: Authorizing `identity:get_project(project_id=82238b119c5840de9fc817856cec36fb)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:58.655631 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51ad8f7-4e61-445f-b0fd-101925dcd9d3 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:58.657320 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 170/360] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Aug 31 01:03:58 2022] GET /identity/v3/projects/82238b119c5840de9fc817856cec36fb => generated 298 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:03:58.658105 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51ad8f7-4e61-445f-b0fd-101925dcd9d3 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:58.715685 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:58.748483 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4364cee4-1756-429d-a8ef-03a4f1b644f9 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:58.780123 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51ad8f7-4e61-445f-b0fd-101925dcd9d3 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:58.786870 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc8b5c4-9ba7-45a6-9e38-d14a5ce58940 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:58.787508 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc8b5c4-9ba7-45a6-9e38-d14a5ce58940 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:58.788228 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc8b5c4-9ba7-45a6-9e38-d14a5ce58940 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:58.788514 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc8b5c4-9ba7-45a6-9e38-d14a5ce58940 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:58.797307 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:58.801133 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc8b5c4-9ba7-45a6-9e38-d14a5ce58940 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 01:03:58.806600 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application [None req-7dc8b5c4-9ba7-45a6-9e38-d14a5ce58940 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 01:03:58.806600 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:03:58.806600 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:03:58.806600 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:03:58.806600 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:03:58.806600 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:03:58.806600 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:03:58.806600 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:03:58.806600 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:03:58.806600 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:03:58.806600 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:03:58.806600 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:03:58.806600 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 01:03:58.806600 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 01:03:58.806600 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 01:03:58.806600 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 01:03:58.807497 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 01:03:58.807497 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 01:03:58.807497 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 01:03:58.807497 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 01:03:58.807497 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:03:58.807497 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:03:58.807497 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:03:58.807497 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:03:58.807497 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:03:58.807497 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:03:58.807497 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:03:58.807497 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application with Lock( Aug 31 01:03:58.807497 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:03:58.807497 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:03:58.807497 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:03:58.807497 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:03:58.808324 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:03:58.808324 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:03:58.808324 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:03:58.808324 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:03:58.808324 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 01:03:58.808324 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 01:03:58.808324 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 01:03:58.808324 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 01:03:58.808324 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 01:03:58.808324 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 01:03:58.808324 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 01:03:58.808324 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Aug 31 01:03:58.808324 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 191/361] 10.222.0.20 () {56 vars in 1114 bytes} [Wed Aug 31 01:03:58 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:03:58.809431 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc02553b-9ac0-4941-a174-96fdd92836c4 None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:58.809711 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc02553b-9ac0-4941-a174-96fdd92836c4 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:58.809982 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc02553b-9ac0-4941-a174-96fdd92836c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:58.827209 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d9271d-50cc-4087-8bac-fbf32f289f6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:58.828114 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d9271d-50cc-4087-8bac-fbf32f289f6e None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:58.828419 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d9271d-50cc-4087-8bac-fbf32f289f6e None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:58.828791 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d9271d-50cc-4087-8bac-fbf32f289f6e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:58.829407 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-fc02553b-9ac0-4941-a174-96fdd92836c4 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:03:58.829540 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d9271d-50cc-4087-8bac-fbf32f289f6e None admin] RBAC: Adding query filter params (role.id=e1aef2df18ab425a8b78190b0fbfe8ba, user.id=df387b9f2fa6483eb82d2b0934a9a274, scope.system=all) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:03:58.829800 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d9271d-50cc-4087-8bac-fbf32f289f6e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:58.831271 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d9271d-50cc-4087-8bac-fbf32f289f6e None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:58.831587 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-fc02553b-9ac0-4941-a174-96fdd92836c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:03:58.846772 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 171/362] 10.222.0.20 () {56 vars in 1308 bytes} [Wed Aug 31 01:03:58 2022] GET /identity/v3/role_assignments?user.id=df387b9f2fa6483eb82d2b0934a9a274&scope.system=all&role.id=e1aef2df18ab425a8b78190b0fbfe8ba => generated 510 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:03:58.847746 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d9271d-50cc-4087-8bac-fbf32f289f6e None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:58.933352 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 192/363] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:03:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 01:03:58.934065 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc02553b-9ac0-4941-a174-96fdd92836c4 None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:58.992531 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d9271d-50cc-4087-8bac-fbf32f289f6e None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:59.005194 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:59.040821 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4c62c9-48f9-4638-9c35-84ec9c508cbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:59.041466 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4c62c9-48f9-4638-9c35-84ec9c508cbe None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:59.041749 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4c62c9-48f9-4638-9c35-84ec9c508cbe None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:59.042023 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4c62c9-48f9-4638-9c35-84ec9c508cbe None admin] PATH_INFO: `/v3/projects/82238b119c5840de9fc817856cec36fb` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:59.043145 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4c62c9-48f9-4638-9c35-84ec9c508cbe None admin] RBAC: Authorizing `identity:get_project(project_id=82238b119c5840de9fc817856cec36fb)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:59.044614 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4c62c9-48f9-4638-9c35-84ec9c508cbe None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:59.046699 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 172/364] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Aug 31 01:03:58 2022] GET /identity/v3/projects/82238b119c5840de9fc817856cec36fb => generated 298 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:03:59.047489 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4c62c9-48f9-4638-9c35-84ec9c508cbe None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:59.063770 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc02553b-9ac0-4941-a174-96fdd92836c4 None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:59.096979 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:59.107631 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7a9eef-9384-4222-9aab-a7f09e862383 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:59.108250 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7a9eef-9384-4222-9aab-a7f09e862383 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:59.109744 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7a9eef-9384-4222-9aab-a7f09e862383 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:59.109744 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7a9eef-9384-4222-9aab-a7f09e862383 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:59.109744 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7a9eef-9384-4222-9aab-a7f09e862383 None admin] RBAC: Adding query filter params (group.id=73043087b04d4917b9fdcbd262ef56e0, role.id=d0b9768f4d014766b9326387f2e11f34, scope.project.id=b4ae9d03e35549cfbadf1d1001837bb6) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:03:59.110162 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7a9eef-9384-4222-9aab-a7f09e862383 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:59.111465 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7a9eef-9384-4222-9aab-a7f09e862383 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:59.118505 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 193/365] 10.222.0.20 () {56 vars in 1376 bytes} [Wed Aug 31 01:03:58 2022] GET /identity/v3/role_assignments?group.id=73043087b04d4917b9fdcbd262ef56e0&scope.project.id=b4ae9d03e35549cfbadf1d1001837bb6&role.id=d0b9768f4d014766b9326387f2e11f34 => generated 260 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:03:59.119321 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7a9eef-9384-4222-9aab-a7f09e862383 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:59.185581 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4c62c9-48f9-4638-9c35-84ec9c508cbe None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:59.224016 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd305e2-d175-498c-b94c-1f5692ad9a0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:59.224777 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd305e2-d175-498c-b94c-1f5692ad9a0e None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:59.224967 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd305e2-d175-498c-b94c-1f5692ad9a0e None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:59.225243 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd305e2-d175-498c-b94c-1f5692ad9a0e None admin] PATH_INFO: `/v3/groups/73043087b04d4917b9fdcbd262ef56e0` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:59.226388 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd305e2-d175-498c-b94c-1f5692ad9a0e None admin] RBAC: Authorizing `identity:get_group(group_id=73043087b04d4917b9fdcbd262ef56e0)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:59.227965 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd305e2-d175-498c-b94c-1f5692ad9a0e None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:59.229619 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 173/366] 10.222.0.20 () {56 vars in 1180 bytes} [Wed Aug 31 01:03:59 2022] GET /identity/v3/groups/73043087b04d4917b9fdcbd262ef56e0 => generated 226 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:03:59.230409 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd305e2-d175-498c-b94c-1f5692ad9a0e None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:59.263228 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7a9eef-9384-4222-9aab-a7f09e862383 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:59.291802 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:59.307932 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905406ea-dfef-4927-be0f-13b16e58b797 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:59.308552 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905406ea-dfef-4927-be0f-13b16e58b797 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:59.308866 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905406ea-dfef-4927-be0f-13b16e58b797 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:59.309144 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905406ea-dfef-4927-be0f-13b16e58b797 None admin] PATH_INFO: `/v3/roles` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:59.309812 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905406ea-dfef-4927-be0f-13b16e58b797 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:03:59.310120 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905406ea-dfef-4927-be0f-13b16e58b797 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:59.311505 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905406ea-dfef-4927-be0f-13b16e58b797 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:59.320561 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 194/367] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Aug 31 01:03:59 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:03:59.321823 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905406ea-dfef-4927-be0f-13b16e58b797 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:59.359956 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd305e2-d175-498c-b94c-1f5692ad9a0e None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:59.395699 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b0414d-16fa-4807-aa11-49421a6c536d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:59.396343 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b0414d-16fa-4807-aa11-49421a6c536d None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:59.396628 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b0414d-16fa-4807-aa11-49421a6c536d None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:59.396934 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b0414d-16fa-4807-aa11-49421a6c536d None admin] PATH_INFO: `/v3/groups/73043087b04d4917b9fdcbd262ef56e0` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:59.398049 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b0414d-16fa-4807-aa11-49421a6c536d None admin] RBAC: Authorizing `identity:get_group(group_id=73043087b04d4917b9fdcbd262ef56e0)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:59.399550 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b0414d-16fa-4807-aa11-49421a6c536d None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:59.400602 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:59.406173 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 174/368] 10.222.0.20 () {56 vars in 1180 bytes} [Wed Aug 31 01:03:59 2022] GET /identity/v3/groups/73043087b04d4917b9fdcbd262ef56e0 => generated 226 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:03:59.406173 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b0414d-16fa-4807-aa11-49421a6c536d None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:59.466588 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905406ea-dfef-4927-be0f-13b16e58b797 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:59.512322 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce57fe46-e1ed-4e80-84ff-c7171bba94af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:59.512853 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce57fe46-e1ed-4e80-84ff-c7171bba94af None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:59.513137 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce57fe46-e1ed-4e80-84ff-c7171bba94af None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:59.513462 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce57fe46-e1ed-4e80-84ff-c7171bba94af None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:59.514136 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce57fe46-e1ed-4e80-84ff-c7171bba94af None admin] RBAC: Adding query filter params (group.id=73043087b04d4917b9fdcbd262ef56e0, role.id=d0b9768f4d014766b9326387f2e11f34, scope.project.id=82238b119c5840de9fc817856cec36fb) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:03:59.514446 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce57fe46-e1ed-4e80-84ff-c7171bba94af None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:59.515892 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce57fe46-e1ed-4e80-84ff-c7171bba94af None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:59.524831 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 195/369] 10.222.0.20 () {56 vars in 1376 bytes} [Wed Aug 31 01:03:59 2022] GET /identity/v3/role_assignments?group.id=73043087b04d4917b9fdcbd262ef56e0&scope.project.id=82238b119c5840de9fc817856cec36fb&role.id=d0b9768f4d014766b9326387f2e11f34 => generated 260 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:03:59.524831 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce57fe46-e1ed-4e80-84ff-c7171bba94af None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:59.539591 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b0414d-16fa-4807-aa11-49421a6c536d None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:59.550951 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:59.587333 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0273f688-0c88-43b6-8925-c6f855696c9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:59.587990 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0273f688-0c88-43b6-8925-c6f855696c9d None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:59.588269 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0273f688-0c88-43b6-8925-c6f855696c9d None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:59.588541 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0273f688-0c88-43b6-8925-c6f855696c9d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:59.605018 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0273f688-0c88-43b6-8925-c6f855696c9d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 01:03:59.612084 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application [None req-0273f688-0c88-43b6-8925-c6f855696c9d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 01:03:59.612084 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:03:59.612084 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:03:59.612084 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:03:59.612084 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:03:59.612084 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:03:59.612084 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:03:59.612084 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:03:59.612084 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:03:59.612084 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:03:59.612084 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:03:59.612084 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:03:59.612084 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 01:03:59.612084 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 01:03:59.612084 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 01:03:59.612084 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 01:03:59.613268 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 01:03:59.613268 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 01:03:59.613268 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 01:03:59.613268 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 01:03:59.613268 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:03:59.613268 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:03:59.613268 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:03:59.613268 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:03:59.613268 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:03:59.613268 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:03:59.613268 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:03:59.613268 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application with Lock( Aug 31 01:03:59.613268 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:03:59.613268 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:03:59.613268 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:03:59.613268 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:03:59.614136 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:03:59.614136 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:03:59.614136 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:03:59.614136 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:03:59.614136 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 01:03:59.614136 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 01:03:59.614136 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 01:03:59.614136 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 01:03:59.614136 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 01:03:59.614136 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 01:03:59.614136 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 01:03:59.614136 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Aug 31 01:03:59.614136 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 175/370] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Aug 31 01:03:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:03:59.614136 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0273f688-0c88-43b6-8925-c6f855696c9d None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:59.675920 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce57fe46-e1ed-4e80-84ff-c7171bba94af None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:59.699883 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:59.721678 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb7b2b9-7629-448d-a2f6-4243f407bbae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:59.724330 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb7b2b9-7629-448d-a2f6-4243f407bbae None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:59.724330 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb7b2b9-7629-448d-a2f6-4243f407bbae None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:59.724330 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb7b2b9-7629-448d-a2f6-4243f407bbae None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:59.738980 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0273f688-0c88-43b6-8925-c6f855696c9d None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:59.779934 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad7a6b7-f4a8-4b8d-919e-5207df3467a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:59.781110 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad7a6b7-f4a8-4b8d-919e-5207df3467a7 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:59.781646 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad7a6b7-f4a8-4b8d-919e-5207df3467a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:59.782193 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad7a6b7-f4a8-4b8d-919e-5207df3467a7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:59.783987 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad7a6b7-f4a8-4b8d-919e-5207df3467a7 None admin] RBAC: Adding query filter params (group.id=73043087b04d4917b9fdcbd262ef56e0, role.id=bebc77e8dfd54dbba7079f93218e7c7d, scope.project.id=82238b119c5840de9fc817856cec36fb) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:03:59.784585 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad7a6b7-f4a8-4b8d-919e-5207df3467a7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:59.786631 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad7a6b7-f4a8-4b8d-919e-5207df3467a7 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:59.794481 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 176/371] 10.222.0.20 () {56 vars in 1376 bytes} [Wed Aug 31 01:03:59 2022] GET /identity/v3/role_assignments?group.id=73043087b04d4917b9fdcbd262ef56e0&scope.project.id=82238b119c5840de9fc817856cec36fb&role.id=bebc77e8dfd54dbba7079f93218e7c7d => generated 260 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:03:59.795638 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad7a6b7-f4a8-4b8d-919e-5207df3467a7 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:59.814797 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb7b2b9-7629-448d-a2f6-4243f407bbae None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:59.816284 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb7b2b9-7629-448d-a2f6-4243f407bbae None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:59.857480 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:59.944968 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 196/372] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Aug 31 01:03:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 01:03:59.951051 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb7b2b9-7629-448d-a2f6-4243f407bbae None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:59.977113 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:00.083589 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad7a6b7-f4a8-4b8d-919e-5207df3467a7 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:00.083867 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb7b2b9-7629-448d-a2f6-4243f407bbae None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:00.107134 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbfad33-89b5-4bdf-91fb-b8b8a568c401 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:00.107776 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbfad33-89b5-4bdf-91fb-b8b8a568c401 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:00.108062 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbfad33-89b5-4bdf-91fb-b8b8a568c401 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:00.108339 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbfad33-89b5-4bdf-91fb-b8b8a568c401 None admin] PATH_INFO: `/v3/projects/b4ae9d03e35549cfbadf1d1001837bb6` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:00.109616 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbfad33-89b5-4bdf-91fb-b8b8a568c401 None admin] RBAC: Authorizing `identity:get_project(project_id=b4ae9d03e35549cfbadf1d1001837bb6)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:00.111167 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbfad33-89b5-4bdf-91fb-b8b8a568c401 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:00.113169 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 197/373] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Aug 31 01:03:59 2022] GET /identity/v3/projects/b4ae9d03e35549cfbadf1d1001837bb6 => generated 302 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 01:04:00.118879 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbfad33-89b5-4bdf-91fb-b8b8a568c401 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:00.177731 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134a5992-6861-4da7-a06f-85f5a0a43799 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:00.178628 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134a5992-6861-4da7-a06f-85f5a0a43799 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:00.179121 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134a5992-6861-4da7-a06f-85f5a0a43799 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:00.179553 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134a5992-6861-4da7-a06f-85f5a0a43799 None admin] PATH_INFO: `/v3/roles` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:00.180517 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134a5992-6861-4da7-a06f-85f5a0a43799 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:04:00.181099 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134a5992-6861-4da7-a06f-85f5a0a43799 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:00.182649 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134a5992-6861-4da7-a06f-85f5a0a43799 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:00.201350 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 177/374] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Aug 31 01:03:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 406 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:04:00.207535 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134a5992-6861-4da7-a06f-85f5a0a43799 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:00.217245 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:00.226122 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbfad33-89b5-4bdf-91fb-b8b8a568c401 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:00.262278 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fa7056-2f71-4b1e-aede-581508b1220e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:00.262945 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fa7056-2f71-4b1e-aede-581508b1220e None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:00.263226 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fa7056-2f71-4b1e-aede-581508b1220e None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:00.263499 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fa7056-2f71-4b1e-aede-581508b1220e None admin] PATH_INFO: `/v3/groups/73043087b04d4917b9fdcbd262ef56e0` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:00.264618 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fa7056-2f71-4b1e-aede-581508b1220e None admin] RBAC: Authorizing `identity:get_group(group_id=73043087b04d4917b9fdcbd262ef56e0)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:00.266373 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fa7056-2f71-4b1e-aede-581508b1220e None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:00.268449 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 198/375] 10.222.0.20 () {56 vars in 1180 bytes} [Wed Aug 31 01:04:00 2022] GET /identity/v3/groups/73043087b04d4917b9fdcbd262ef56e0 => generated 226 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:04:00.277343 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fa7056-2f71-4b1e-aede-581508b1220e None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:00.386336 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fa7056-2f71-4b1e-aede-581508b1220e None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:00.423845 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d36a0c-f9ff-4a40-badb-fdd94df4461b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:00.424465 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d36a0c-f9ff-4a40-badb-fdd94df4461b None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:00.424763 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d36a0c-f9ff-4a40-badb-fdd94df4461b None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:00.425043 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d36a0c-f9ff-4a40-badb-fdd94df4461b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:00.425769 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d36a0c-f9ff-4a40-badb-fdd94df4461b None admin] RBAC: Adding query filter params (group.id=73043087b04d4917b9fdcbd262ef56e0, role.id=bebc77e8dfd54dbba7079f93218e7c7d, scope.project.id=b4ae9d03e35549cfbadf1d1001837bb6) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:04:00.426079 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d36a0c-f9ff-4a40-badb-fdd94df4461b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:00.427521 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d36a0c-f9ff-4a40-badb-fdd94df4461b None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:00.433466 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 199/376] 10.222.0.20 () {56 vars in 1376 bytes} [Wed Aug 31 01:04:00 2022] GET /identity/v3/role_assignments?group.id=73043087b04d4917b9fdcbd262ef56e0&scope.project.id=b4ae9d03e35549cfbadf1d1001837bb6&role.id=bebc77e8dfd54dbba7079f93218e7c7d => generated 260 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 01:04:00.442791 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:00.455302 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134a5992-6861-4da7-a06f-85f5a0a43799 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:00.503826 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25027a8e-1d8d-4653-bb41-34d4927b42ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:00.504446 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25027a8e-1d8d-4653-bb41-34d4927b42ab None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:00.505020 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25027a8e-1d8d-4653-bb41-34d4927b42ab None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:00.505296 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25027a8e-1d8d-4653-bb41-34d4927b42ab None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:00.612423 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25027a8e-1d8d-4653-bb41-34d4927b42ab None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:00.613962 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25027a8e-1d8d-4653-bb41-34d4927b42ab None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:00.733764 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:00.741818 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 178/377] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Aug 31 01:04:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 535 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 01:04:00.747511 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d36a0c-f9ff-4a40-badb-fdd94df4461b None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:00.793103 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:00.908451 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d36a0c-f9ff-4a40-badb-fdd94df4461b None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:00.961426 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dc4b2f-49c6-4f2d-80b7-09424596b23d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:00.962097 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dc4b2f-49c6-4f2d-80b7-09424596b23d None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:00.962379 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dc4b2f-49c6-4f2d-80b7-09424596b23d None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:00.962650 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dc4b2f-49c6-4f2d-80b7-09424596b23d None admin] PATH_INFO: `/v3/projects/77c8e2846a11431eb786ad3a5192ea92` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:00.969406 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2dc4b2f-49c6-4f2d-80b7-09424596b23d None admin] RBAC: Authorizing `identity:get_project(project_id=77c8e2846a11431eb786ad3a5192ea92)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:00.970855 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2dc4b2f-49c6-4f2d-80b7-09424596b23d None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:00.972672 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 200/378] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Aug 31 01:04:00 2022] GET /identity/v3/projects/77c8e2846a11431eb786ad3a5192ea92 => generated 344 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:04:00.978065 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25027a8e-1d8d-4653-bb41-34d4927b42ab None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:01.124352 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25027a8e-1d8d-4653-bb41-34d4927b42ab None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:01.175270 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52de06c-cb3a-4940-91dc-4cd743a8e7e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:01.175907 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52de06c-cb3a-4940-91dc-4cd743a8e7e8 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:01.176185 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52de06c-cb3a-4940-91dc-4cd743a8e7e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:01.176459 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52de06c-cb3a-4940-91dc-4cd743a8e7e8 None admin] PATH_INFO: `/v3/groups/10f6285d94084e98940bda76f4eaf253` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:01.187324 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52de06c-cb3a-4940-91dc-4cd743a8e7e8 None admin] RBAC: Authorizing `identity:get_group(group_id=10f6285d94084e98940bda76f4eaf253)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:01.188817 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52de06c-cb3a-4940-91dc-4cd743a8e7e8 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:01.190647 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 179/379] 10.222.0.20 () {56 vars in 1180 bytes} [Wed Aug 31 01:04:00 2022] GET /identity/v3/groups/10f6285d94084e98940bda76f4eaf253 => generated 229 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:04:01.196299 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dc4b2f-49c6-4f2d-80b7-09424596b23d None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:01.324194 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:01.336865 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2072a93-a73e-4676-b0d1-59d3af9bdcd3 None None] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:01.337348 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2072a93-a73e-4676-b0d1-59d3af9bdcd3 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:01.346637 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2072a93-a73e-4676-b0d1-59d3af9bdcd3 None None] PATH_INFO: `/` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:01.346637 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 180/380] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:04:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 01:04:01.346812 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dc4b2f-49c6-4f2d-80b7-09424596b23d None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:01.346909 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed4d4e2-27f8-4fb3-9a27-602098760036 None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:01.346909 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed4d4e2-27f8-4fb3-9a27-602098760036 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:01.346909 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed4d4e2-27f8-4fb3-9a27-602098760036 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:01.368077 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-5ed4d4e2-27f8-4fb3-9a27-602098760036 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:04:01.370040 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-5ed4d4e2-27f8-4fb3-9a27-602098760036 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:04:01.398642 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1185d280-15bf-4424-9cac-673703e9b332 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:01.399243 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1185d280-15bf-4424-9cac-673703e9b332 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:01.399532 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1185d280-15bf-4424-9cac-673703e9b332 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:01.399815 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1185d280-15bf-4424-9cac-673703e9b332 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:01.400684 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1185d280-15bf-4424-9cac-673703e9b332 None admin] RBAC: Adding query filter params (group.id=10f6285d94084e98940bda76f4eaf253, role.id=d009411496b8452b88e77ffb8db015a7, scope.project.id=77c8e2846a11431eb786ad3a5192ea92) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:04:01.401350 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1185d280-15bf-4424-9cac-673703e9b332 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:01.403173 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1185d280-15bf-4424-9cac-673703e9b332 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:01.415324 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 201/381] 10.222.0.20 () {56 vars in 1376 bytes} [Wed Aug 31 01:04:01 2022] GET /identity/v3/role_assignments?group.id=10f6285d94084e98940bda76f4eaf253&scope.project.id=77c8e2846a11431eb786ad3a5192ea92&role.id=d009411496b8452b88e77ffb8db015a7 => generated 260 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:04:01.454178 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 181/382] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:04:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 01:04:01.454715 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:01.800032 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1185d280-15bf-4424-9cac-673703e9b332 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:01.904144 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523ca0be-c8ba-4783-8fbe-61f918e979a8 None None] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:01.904446 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523ca0be-c8ba-4783-8fbe-61f918e979a8 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:01.904714 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523ca0be-c8ba-4783-8fbe-61f918e979a8 None None] PATH_INFO: `/` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:01.906829 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 182/383] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:04:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:04:01.911774 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b9557e-52cc-4cb9-9718-f5f69e21a0ad None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:01.912060 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b9557e-52cc-4cb9-9718-f5f69e21a0ad None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:01.912341 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b9557e-52cc-4cb9-9718-f5f69e21a0ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:01.935712 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1185d280-15bf-4424-9cac-673703e9b332 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:01.952981 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-59b9557e-52cc-4cb9-9718-f5f69e21a0ad None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:04:01.954178 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-59b9557e-52cc-4cb9-9718-f5f69e21a0ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:04:01.980829 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236193a2-262c-43e5-9411-e5d721d6cab9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:01.981464 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236193a2-262c-43e5-9411-e5d721d6cab9 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:01.981738 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236193a2-262c-43e5-9411-e5d721d6cab9 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:01.982829 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236193a2-262c-43e5-9411-e5d721d6cab9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:01.994652 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236193a2-262c-43e5-9411-e5d721d6cab9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 01:04:01.999844 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application [None req-236193a2-262c-43e5-9411-e5d721d6cab9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 01:04:01.999844 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:04:01.999844 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:04:01.999844 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:04:01.999844 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:04:01.999844 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:04:01.999844 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:04:01.999844 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:04:01.999844 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:04:01.999844 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:04:01.999844 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:04:01.999844 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:04:01.999844 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 01:04:01.999844 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 01:04:01.999844 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 01:04:01.999844 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 01:04:02.000831 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 01:04:02.000831 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 01:04:02.000831 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 01:04:02.000831 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 01:04:02.000831 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:04:02.000831 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:04:02.000831 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:04:02.000831 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:04:02.000831 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:04:02.000831 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:04:02.000831 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:04:02.000831 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application with Lock( Aug 31 01:04:02.000831 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:04:02.000831 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:04:02.000831 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:04:02.000831 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:04:02.001662 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:04:02.001662 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:04:02.001662 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:04:02.001662 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:04:02.001662 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 01:04:02.001662 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 01:04:02.001662 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 01:04:02.001662 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 01:04:02.001662 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 01:04:02.001662 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 01:04:02.001662 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 01:04:02.001662 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Aug 31 01:04:02.005391 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 202/384] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Aug 31 01:04:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:04:02.009616 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236193a2-262c-43e5-9411-e5d721d6cab9 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:02.037598 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:02.091988 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 183/385] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:04:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 01:04:02.174854 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236193a2-262c-43e5-9411-e5d721d6cab9 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:02.176207 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b8aa29-c8ff-49e9-8ef4-d6dd93f073a1 None None] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:02.176497 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b8aa29-c8ff-49e9-8ef4-d6dd93f073a1 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:02.176811 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b8aa29-c8ff-49e9-8ef4-d6dd93f073a1 None None] PATH_INFO: `/` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:02.178282 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 184/386] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:04:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 01:04:02.191394 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd3aa7a-d672-454c-9b76-ba2cfc74d0a4 None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:02.191680 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd3aa7a-d672-454c-9b76-ba2cfc74d0a4 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:02.191956 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd3aa7a-d672-454c-9b76-ba2cfc74d0a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:02.223318 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add5c2a7-0f4b-4148-b308-3f6b9b6ba87d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:02.224676 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add5c2a7-0f4b-4148-b308-3f6b9b6ba87d None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:02.228184 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add5c2a7-0f4b-4148-b308-3f6b9b6ba87d None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:02.228674 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add5c2a7-0f4b-4148-b308-3f6b9b6ba87d None admin] PATH_INFO: `/v3/roles` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:02.229569 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add5c2a7-0f4b-4148-b308-3f6b9b6ba87d None admin] RBAC: Adding query filter params (name=member) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:04:02.230085 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add5c2a7-0f4b-4148-b308-3f6b9b6ba87d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:02.231651 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add5c2a7-0f4b-4148-b308-3f6b9b6ba87d None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:02.232820 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-ffd3aa7a-d672-454c-9b76-ba2cfc74d0a4 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:04:02.235335 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-ffd3aa7a-d672-454c-9b76-ba2cfc74d0a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:04:02.249481 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 203/387] 10.222.0.20 () {56 vars in 1101 bytes} [Wed Aug 31 01:04:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:04:02.256334 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add5c2a7-0f4b-4148-b308-3f6b9b6ba87d None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:02.342581 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 185/388] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:04:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 01:04:02.389602 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:02.424345 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd3aa7a-d672-454c-9b76-ba2cfc74d0a4 None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:02.424849 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add5c2a7-0f4b-4148-b308-3f6b9b6ba87d None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:02.465402 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3d03f4-ca1f-405a-a5f7-6b5637bd0626 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:02.466032 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3d03f4-ca1f-405a-a5f7-6b5637bd0626 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:02.466314 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3d03f4-ca1f-405a-a5f7-6b5637bd0626 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:02.466586 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3d03f4-ca1f-405a-a5f7-6b5637bd0626 None admin] PATH_INFO: `/v3/groups/73043087b04d4917b9fdcbd262ef56e0` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:02.468568 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3d03f4-ca1f-405a-a5f7-6b5637bd0626 None admin] RBAC: Authorizing `identity:get_group(group_id=73043087b04d4917b9fdcbd262ef56e0)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:02.470025 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3d03f4-ca1f-405a-a5f7-6b5637bd0626 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:02.471799 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 204/389] 10.222.0.20 () {56 vars in 1180 bytes} [Wed Aug 31 01:04:02 2022] GET /identity/v3/groups/73043087b04d4917b9fdcbd262ef56e0 => generated 226 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:04:02.478628 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3d03f4-ca1f-405a-a5f7-6b5637bd0626 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:02.525144 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:02.574169 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd3aa7a-d672-454c-9b76-ba2cfc74d0a4 None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:02.638785 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0d1f7d-4e4c-4c06-9bf6-ff07f0e31751 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:02.639400 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0d1f7d-4e4c-4c06-9bf6-ff07f0e31751 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:02.639730 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0d1f7d-4e4c-4c06-9bf6-ff07f0e31751 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:02.640006 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0d1f7d-4e4c-4c06-9bf6-ff07f0e31751 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:02.659268 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0d1f7d-4e4c-4c06-9bf6-ff07f0e31751 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 01:04:02.665811 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application [None req-ce0d1f7d-4e4c-4c06-9bf6-ff07f0e31751 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 01:04:02.665811 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:04:02.665811 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:04:02.665811 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:04:02.665811 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:04:02.665811 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:04:02.665811 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:04:02.665811 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:04:02.665811 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:04:02.665811 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:04:02.665811 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:04:02.665811 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:04:02.665811 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 01:04:02.665811 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 01:04:02.665811 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 01:04:02.665811 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 01:04:02.666716 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 01:04:02.666716 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 01:04:02.666716 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 01:04:02.666716 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 01:04:02.666716 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:04:02.666716 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:04:02.666716 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:04:02.666716 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:04:02.666716 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:04:02.666716 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:04:02.666716 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:04:02.666716 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application with Lock( Aug 31 01:04:02.666716 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:04:02.666716 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:04:02.666716 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:04:02.666716 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:04:02.668114 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:04:02.668114 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:04:02.668114 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:04:02.668114 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:04:02.668114 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 01:04:02.668114 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 01:04:02.668114 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 01:04:02.668114 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 01:04:02.668114 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 01:04:02.668114 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 01:04:02.668114 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 01:04:02.668114 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Aug 31 01:04:02.668712 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 186/390] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Aug 31 01:04:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 01:04:02.672577 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0d1f7d-4e4c-4c06-9bf6-ff07f0e31751 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:02.687073 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3d03f4-ca1f-405a-a5f7-6b5637bd0626 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:02.733931 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b5709e-e9d2-4228-a74e-765816c2a090 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:02.734564 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b5709e-e9d2-4228-a74e-765816c2a090 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:02.734842 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b5709e-e9d2-4228-a74e-765816c2a090 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:02.735112 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b5709e-e9d2-4228-a74e-765816c2a090 None admin] PATH_INFO: `/v3/projects/b4ae9d03e35549cfbadf1d1001837bb6` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:02.736235 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b5709e-e9d2-4228-a74e-765816c2a090 None admin] RBAC: Authorizing `identity:get_project(project_id=b4ae9d03e35549cfbadf1d1001837bb6)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:02.737708 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b5709e-e9d2-4228-a74e-765816c2a090 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:02.739617 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 205/391] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Aug 31 01:04:02 2022] GET /identity/v3/projects/b4ae9d03e35549cfbadf1d1001837bb6 => generated 302 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 01:04:02.740386 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b5709e-e9d2-4228-a74e-765816c2a090 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:02.753496 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:02.826839 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0d1f7d-4e4c-4c06-9bf6-ff07f0e31751 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:02.838286 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:02.854347 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b5709e-e9d2-4228-a74e-765816c2a090 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:02.865015 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ebf554-e661-45cf-9b59-3615b2fa0792 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:02.865942 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ebf554-e661-45cf-9b59-3615b2fa0792 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:02.866468 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ebf554-e661-45cf-9b59-3615b2fa0792 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:02.866667 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ebf554-e661-45cf-9b59-3615b2fa0792 None admin] PATH_INFO: `/v3/roles` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:02.867337 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ebf554-e661-45cf-9b59-3615b2fa0792 None admin] RBAC: Adding query filter params (name=member) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:04:02.867647 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ebf554-e661-45cf-9b59-3615b2fa0792 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:02.869933 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ebf554-e661-45cf-9b59-3615b2fa0792 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:02.878929 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 187/392] 10.222.0.20 () {56 vars in 1101 bytes} [Wed Aug 31 01:04:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:04:02.884008 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ebf554-e661-45cf-9b59-3615b2fa0792 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:02.890419 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa5cd62-83c5-45cb-94cd-81bb56a79574 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:02.891000 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa5cd62-83c5-45cb-94cd-81bb56a79574 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:02.891274 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa5cd62-83c5-45cb-94cd-81bb56a79574 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:02.896926 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa5cd62-83c5-45cb-94cd-81bb56a79574 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:02.908230 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa5cd62-83c5-45cb-94cd-81bb56a79574 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 01:04:02.913720 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application [None req-caa5cd62-83c5-45cb-94cd-81bb56a79574 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 01:04:02.913720 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:04:02.913720 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:04:02.913720 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:04:02.913720 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:04:02.913720 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:04:02.913720 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:04:02.913720 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:04:02.913720 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:04:02.913720 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:04:02.913720 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:04:02.913720 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:04:02.913720 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 01:04:02.913720 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 01:04:02.913720 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 01:04:02.913720 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 01:04:02.915199 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 01:04:02.915199 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 01:04:02.915199 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 01:04:02.915199 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 01:04:02.915199 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:04:02.915199 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:04:02.915199 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:04:02.915199 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:04:02.915199 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:04:02.915199 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:04:02.915199 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:04:02.915199 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application with Lock( Aug 31 01:04:02.915199 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:04:02.915199 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:04:02.915199 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:04:02.915199 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:04:02.916046 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:04:02.916046 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:04:02.916046 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:04:02.916046 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:04:02.916046 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 01:04:02.916046 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 01:04:02.916046 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 01:04:02.916046 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 01:04:02.916046 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 01:04:02.916046 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 01:04:02.916046 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 01:04:02.916046 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Aug 31 01:04:02.916046 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 206/393] 10.222.0.20 () {56 vars in 1114 bytes} [Wed Aug 31 01:04:02 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:04:02.917886 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36501ed5-a72a-4d85-ac06-4667f132f6a2 None None] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:02.918165 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36501ed5-a72a-4d85-ac06-4667f132f6a2 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:02.918436 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36501ed5-a72a-4d85-ac06-4667f132f6a2 None None] PATH_INFO: `/` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:02.919836 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 207/394] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:04:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:04:02.920424 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36501ed5-a72a-4d85-ac06-4667f132f6a2 None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:02.995795 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:03.000684 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ebf554-e661-45cf-9b59-3615b2fa0792 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:03.033511 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36501ed5-a72a-4d85-ac06-4667f132f6a2 None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:03.035154 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cb4ee4-5706-4a67-b9e3-3c136ddada7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:03.036221 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cb4ee4-5706-4a67-b9e3-3c136ddada7b None admin] REQUEST_METHOD: `PUT` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:03.036221 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cb4ee4-5706-4a67-b9e3-3c136ddada7b None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:03.036417 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cb4ee4-5706-4a67-b9e3-3c136ddada7b None admin] PATH_INFO: `/v3/projects/b4ae9d03e35549cfbadf1d1001837bb6/groups/73043087b04d4917b9fdcbd262ef56e0/roles/d0b9768f4d014766b9326387f2e11f34` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:03.038237 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cb4ee4-5706-4a67-b9e3-3c136ddada7b None admin] RBAC: Authorizing `identity:create_grant(project_id=b4ae9d03e35549cfbadf1d1001837bb6, group_id=73043087b04d4917b9fdcbd262ef56e0, role_id=d0b9768f4d014766b9326387f2e11f34)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:03.039787 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cb4ee4-5706-4a67-b9e3-3c136ddada7b None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:03.067207 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0d5b1e-a95f-4edd-870c-843e0a17b2b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:03.067833 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0d5b1e-a95f-4edd-870c-843e0a17b2b8 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:03.068115 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0d5b1e-a95f-4edd-870c-843e0a17b2b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:03.068388 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0d5b1e-a95f-4edd-870c-843e0a17b2b8 None admin] PATH_INFO: `/v3/groups/73043087b04d4917b9fdcbd262ef56e0` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:03.069523 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0d5b1e-a95f-4edd-870c-843e0a17b2b8 None admin] RBAC: Authorizing `identity:get_group(group_id=73043087b04d4917b9fdcbd262ef56e0)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:03.071611 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0d5b1e-a95f-4edd-870c-843e0a17b2b8 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:03.073203 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 208/395] 10.222.0.20 () {56 vars in 1180 bytes} [Wed Aug 31 01:04:02 2022] GET /identity/v3/groups/73043087b04d4917b9fdcbd262ef56e0 => generated 226 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:04:03.074202 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0d5b1e-a95f-4edd-870c-843e0a17b2b8 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:03.108760 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 188/396] 10.222.0.20 () {58 vars in 1442 bytes} [Wed Aug 31 01:04:02 2022] PUT /identity/v3/projects/b4ae9d03e35549cfbadf1d1001837bb6/groups/73043087b04d4917b9fdcbd262ef56e0/roles/d0b9768f4d014766b9326387f2e11f34 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:04:03.110253 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeae9e39-6710-4fa2-bc20-99348f9638cf None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:03.110582 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeae9e39-6710-4fa2-bc20-99348f9638cf None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:03.110864 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeae9e39-6710-4fa2-bc20-99348f9638cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:03.136106 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-eeae9e39-6710-4fa2-bc20-99348f9638cf None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:04:03.137819 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-eeae9e39-6710-4fa2-bc20-99348f9638cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:04:03.168116 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:03.191487 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0d5b1e-a95f-4edd-870c-843e0a17b2b8 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:03.224764 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 189/397] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:04:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 01:04:03.224764 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeae9e39-6710-4fa2-bc20-99348f9638cf None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:03.243637 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6069fda-0079-4c70-b709-26f4a9ea782e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:03.245039 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6069fda-0079-4c70-b709-26f4a9ea782e None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:03.245532 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6069fda-0079-4c70-b709-26f4a9ea782e None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:03.246080 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6069fda-0079-4c70-b709-26f4a9ea782e None admin] PATH_INFO: `/v3/roles` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:03.247035 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6069fda-0079-4c70-b709-26f4a9ea782e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:04:03.252748 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6069fda-0079-4c70-b709-26f4a9ea782e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:03.254615 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6069fda-0079-4c70-b709-26f4a9ea782e None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:03.261104 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:03.266211 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 209/398] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Aug 31 01:04:03 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:04:03.278474 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6069fda-0079-4c70-b709-26f4a9ea782e None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:03.348982 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeae9e39-6710-4fa2-bc20-99348f9638cf None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:03.387521 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d5fab3-2765-4be8-ab12-f8ec6b204187 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:03.388168 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d5fab3-2765-4be8-ab12-f8ec6b204187 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:03.388506 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d5fab3-2765-4be8-ab12-f8ec6b204187 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:03.388823 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d5fab3-2765-4be8-ab12-f8ec6b204187 None admin] PATH_INFO: `/v3/projects/82238b119c5840de9fc817856cec36fb` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:03.389971 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d5fab3-2765-4be8-ab12-f8ec6b204187 None admin] RBAC: Authorizing `identity:get_project(project_id=82238b119c5840de9fc817856cec36fb)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:03.391425 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d5fab3-2765-4be8-ab12-f8ec6b204187 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:03.393455 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 190/399] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Aug 31 01:04:03 2022] GET /identity/v3/projects/82238b119c5840de9fc817856cec36fb => generated 298 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:04:03.399142 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d5fab3-2765-4be8-ab12-f8ec6b204187 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:03.416518 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6069fda-0079-4c70-b709-26f4a9ea782e None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:03.431053 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:03.487094 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213e5018-6420-44c8-8a05-4bdecb0fcc93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:03.487737 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213e5018-6420-44c8-8a05-4bdecb0fcc93 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:03.488017 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213e5018-6420-44c8-8a05-4bdecb0fcc93 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:03.488345 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213e5018-6420-44c8-8a05-4bdecb0fcc93 None admin] PATH_INFO: `/v3/groups/73043087b04d4917b9fdcbd262ef56e0` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:03.489691 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213e5018-6420-44c8-8a05-4bdecb0fcc93 None admin] RBAC: Authorizing `identity:get_group(group_id=73043087b04d4917b9fdcbd262ef56e0)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:03.492406 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213e5018-6420-44c8-8a05-4bdecb0fcc93 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:03.495118 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 210/400] 10.222.0.20 () {56 vars in 1180 bytes} [Wed Aug 31 01:04:03 2022] GET /identity/v3/groups/73043087b04d4917b9fdcbd262ef56e0 => generated 226 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:04:03.501293 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213e5018-6420-44c8-8a05-4bdecb0fcc93 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:03.555368 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d5fab3-2765-4be8-ab12-f8ec6b204187 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:03.593514 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63be1ba4-2f8c-4580-aa83-8f0f96b82877 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:03.594222 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63be1ba4-2f8c-4580-aa83-8f0f96b82877 None admin] REQUEST_METHOD: `PUT` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:03.594508 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63be1ba4-2f8c-4580-aa83-8f0f96b82877 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:03.594781 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63be1ba4-2f8c-4580-aa83-8f0f96b82877 None admin] PATH_INFO: `/v3/projects/82238b119c5840de9fc817856cec36fb/groups/73043087b04d4917b9fdcbd262ef56e0/roles/d0b9768f4d014766b9326387f2e11f34` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:03.595694 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:03.596807 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63be1ba4-2f8c-4580-aa83-8f0f96b82877 None admin] RBAC: Authorizing `identity:create_grant(project_id=82238b119c5840de9fc817856cec36fb, group_id=73043087b04d4917b9fdcbd262ef56e0, role_id=d0b9768f4d014766b9326387f2e11f34)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:03.598364 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63be1ba4-2f8c-4580-aa83-8f0f96b82877 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:03.632501 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213e5018-6420-44c8-8a05-4bdecb0fcc93 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:03.669024 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee981e4c-4b15-4b45-9cc1-acc5889ebb7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:03.669663 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee981e4c-4b15-4b45-9cc1-acc5889ebb7e None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:03.669951 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee981e4c-4b15-4b45-9cc1-acc5889ebb7e None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:03.670226 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee981e4c-4b15-4b45-9cc1-acc5889ebb7e None admin] PATH_INFO: `/v3/projects/82238b119c5840de9fc817856cec36fb` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:03.671368 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee981e4c-4b15-4b45-9cc1-acc5889ebb7e None admin] RBAC: Authorizing `identity:get_project(project_id=82238b119c5840de9fc817856cec36fb)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:03.672855 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee981e4c-4b15-4b45-9cc1-acc5889ebb7e None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:03.679585 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 211/402] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Aug 31 01:04:03 2022] GET /identity/v3/projects/82238b119c5840de9fc817856cec36fb => generated 298 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:04:03.679585 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee981e4c-4b15-4b45-9cc1-acc5889ebb7e None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:03.680123 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 191/402] 10.222.0.20 () {58 vars in 1442 bytes} [Wed Aug 31 01:04:03 2022] PUT /identity/v3/projects/82238b119c5840de9fc817856cec36fb/groups/73043087b04d4917b9fdcbd262ef56e0/roles/d0b9768f4d014766b9326387f2e11f34 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 01:04:03.685952 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63be1ba4-2f8c-4580-aa83-8f0f96b82877 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:03.797215 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:03.811083 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63be1ba4-2f8c-4580-aa83-8f0f96b82877 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:03.830030 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee981e4c-4b15-4b45-9cc1-acc5889ebb7e None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:03.850859 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a209304a-cd89-4076-a63e-9c884b695611 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:03.851514 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a209304a-cd89-4076-a63e-9c884b695611 None admin] REQUEST_METHOD: `PUT` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:03.851801 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a209304a-cd89-4076-a63e-9c884b695611 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:03.861129 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a209304a-cd89-4076-a63e-9c884b695611 None admin] PATH_INFO: `/v3/projects/82238b119c5840de9fc817856cec36fb/groups/73043087b04d4917b9fdcbd262ef56e0/roles/bebc77e8dfd54dbba7079f93218e7c7d` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:03.861129 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a209304a-cd89-4076-a63e-9c884b695611 None admin] RBAC: Authorizing `identity:create_grant(project_id=82238b119c5840de9fc817856cec36fb, group_id=73043087b04d4917b9fdcbd262ef56e0, role_id=bebc77e8dfd54dbba7079f93218e7c7d)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:03.861129 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a209304a-cd89-4076-a63e-9c884b695611 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:03.869637 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523cc5c5-cc16-478c-ac51-f6615ba5f8d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:03.870285 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523cc5c5-cc16-478c-ac51-f6615ba5f8d6 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:03.870780 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523cc5c5-cc16-478c-ac51-f6615ba5f8d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:03.870901 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523cc5c5-cc16-478c-ac51-f6615ba5f8d6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:03.874042 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:03.883074 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523cc5c5-cc16-478c-ac51-f6615ba5f8d6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 01:04:03.888079 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application [None req-523cc5c5-cc16-478c-ac51-f6615ba5f8d6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 01:04:03.888079 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:04:03.888079 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:04:03.888079 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:04:03.888079 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:04:03.888079 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:04:03.888079 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:04:03.888079 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:04:03.888079 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:04:03.888079 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:04:03.888079 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:04:03.888079 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:04:03.888079 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 01:04:03.888079 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 01:04:03.888079 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 01:04:03.888079 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 01:04:03.890564 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 01:04:03.890564 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 01:04:03.890564 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 01:04:03.890564 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 01:04:03.890564 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:04:03.890564 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:04:03.890564 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:04:03.890564 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:04:03.890564 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:04:03.890564 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:04:03.890564 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:04:03.890564 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application with Lock( Aug 31 01:04:03.890564 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:04:03.890564 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:04:03.890564 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:04:03.890564 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:04:03.891385 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:04:03.891385 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:04:03.891385 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:04:03.891385 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:04:03.891385 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 01:04:03.891385 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 01:04:03.891385 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 01:04:03.891385 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 01:04:03.891385 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 01:04:03.891385 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 01:04:03.891385 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 01:04:03.891385 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Aug 31 01:04:03.891385 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 212/403] 10.222.0.20 () {56 vars in 1114 bytes} [Wed Aug 31 01:04:03 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:04:03.891385 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4022aa1-c45b-482f-ac15-10bde906ac17 None None] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:03.892141 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4022aa1-c45b-482f-ac15-10bde906ac17 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:03.892141 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4022aa1-c45b-482f-ac15-10bde906ac17 None None] PATH_INFO: `/` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:03.893700 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 213/404] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:04:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:04:03.898739 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6646bae-851b-4fe9-8c37-edb6a3565382 None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:03.899543 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6646bae-851b-4fe9-8c37-edb6a3565382 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:03.899821 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6646bae-851b-4fe9-8c37-edb6a3565382 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:03.919016 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-e6646bae-851b-4fe9-8c37-edb6a3565382 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:04:03.920195 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-e6646bae-851b-4fe9-8c37-edb6a3565382 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:04:03.992649 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 214/405] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:04:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 01:04:03.992649 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6646bae-851b-4fe9-8c37-edb6a3565382 None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:04.002644 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 192/406] 10.222.0.20 () {58 vars in 1442 bytes} [Wed Aug 31 01:04:03 2022] PUT /identity/v3/projects/82238b119c5840de9fc817856cec36fb/groups/73043087b04d4917b9fdcbd262ef56e0/roles/bebc77e8dfd54dbba7079f93218e7c7d => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:04:04.152452 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6646bae-851b-4fe9-8c37-edb6a3565382 None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:04.195186 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:04.206797 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfdf6ef-51fd-4fd4-a724-0788cdf9cbb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:04.207414 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfdf6ef-51fd-4fd4-a724-0788cdf9cbb7 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:04.207698 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfdf6ef-51fd-4fd4-a724-0788cdf9cbb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:04.207976 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfdf6ef-51fd-4fd4-a724-0788cdf9cbb7 None admin] PATH_INFO: `/v3/roles` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:04.208651 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfdf6ef-51fd-4fd4-a724-0788cdf9cbb7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:04:04.208983 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfdf6ef-51fd-4fd4-a724-0788cdf9cbb7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:04.210358 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfdf6ef-51fd-4fd4-a724-0788cdf9cbb7 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:04.224632 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 215/407] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Aug 31 01:04:03 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:04:04.231465 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a209304a-cd89-4076-a63e-9c884b695611 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:04.297429 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfdf6ef-51fd-4fd4-a724-0788cdf9cbb7 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:04.378793 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a209304a-cd89-4076-a63e-9c884b695611 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:04.426381 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c50614-8416-46c1-a5ee-881ae4601587 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:04.427234 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfdf6ef-51fd-4fd4-a724-0788cdf9cbb7 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:04.427612 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c50614-8416-46c1-a5ee-881ae4601587 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:04.428084 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c50614-8416-46c1-a5ee-881ae4601587 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:04.428839 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c50614-8416-46c1-a5ee-881ae4601587 None admin] PATH_INFO: `/v3/groups/73043087b04d4917b9fdcbd262ef56e0` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:04.431179 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c50614-8416-46c1-a5ee-881ae4601587 None admin] RBAC: Authorizing `identity:get_group(group_id=73043087b04d4917b9fdcbd262ef56e0)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:04.432865 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c50614-8416-46c1-a5ee-881ae4601587 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:04.441440 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 193/408] 10.222.0.20 () {56 vars in 1180 bytes} [Wed Aug 31 01:04:04 2022] GET /identity/v3/groups/73043087b04d4917b9fdcbd262ef56e0 => generated 226 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:04:04.448390 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c50614-8416-46c1-a5ee-881ae4601587 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:04.470128 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:04.529438 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc41d73-6d40-4790-970a-4deba37961fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:04.530091 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc41d73-6d40-4790-970a-4deba37961fb None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:04.530378 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc41d73-6d40-4790-970a-4deba37961fb None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:04.530658 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc41d73-6d40-4790-970a-4deba37961fb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:04.556897 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c50614-8416-46c1-a5ee-881ae4601587 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:04.561625 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc41d73-6d40-4790-970a-4deba37961fb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 01:04:04.567180 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application [None req-ffc41d73-6d40-4790-970a-4deba37961fb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 01:04:04.567180 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:04:04.567180 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:04:04.567180 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:04:04.567180 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:04:04.567180 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:04:04.567180 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:04:04.567180 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:04:04.567180 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:04:04.567180 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:04:04.567180 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:04:04.567180 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:04:04.567180 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 01:04:04.567180 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 01:04:04.567180 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 01:04:04.567180 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 01:04:04.568057 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 01:04:04.568057 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 01:04:04.568057 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 01:04:04.568057 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 01:04:04.568057 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:04:04.568057 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:04:04.568057 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:04:04.568057 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:04:04.568057 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:04:04.568057 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:04:04.568057 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:04:04.568057 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application with Lock( Aug 31 01:04:04.568057 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:04:04.568057 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:04:04.568057 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:04:04.568057 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:04:04.568883 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:04:04.568883 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:04:04.568883 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:04:04.568883 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:04:04.568883 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 01:04:04.568883 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 01:04:04.568883 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 01:04:04.568883 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 01:04:04.568883 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 01:04:04.568883 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 01:04:04.568883 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 01:04:04.568883 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Aug 31 01:04:04.577161 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 216/409] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Aug 31 01:04:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 01:04:04.590137 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc41d73-6d40-4790-970a-4deba37961fb None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:04.605828 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4016d818-f1d5-43d1-b4da-a09d132b42d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:04.606472 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4016d818-f1d5-43d1-b4da-a09d132b42d8 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:04.606756 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4016d818-f1d5-43d1-b4da-a09d132b42d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:04.607034 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4016d818-f1d5-43d1-b4da-a09d132b42d8 None admin] PATH_INFO: `/v3/projects/b4ae9d03e35549cfbadf1d1001837bb6` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:04.609274 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4016d818-f1d5-43d1-b4da-a09d132b42d8 None admin] RBAC: Authorizing `identity:get_project(project_id=b4ae9d03e35549cfbadf1d1001837bb6)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:04.613587 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4016d818-f1d5-43d1-b4da-a09d132b42d8 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:04.615840 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 194/410] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Aug 31 01:04:04 2022] GET /identity/v3/projects/b4ae9d03e35549cfbadf1d1001837bb6 => generated 302 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 01:04:04.621216 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4016d818-f1d5-43d1-b4da-a09d132b42d8 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:04.693917 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:04.797305 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:04.818907 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc41d73-6d40-4790-970a-4deba37961fb None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:04.822922 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4016d818-f1d5-43d1-b4da-a09d132b42d8 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:04.884302 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f8a094-e094-40b8-8a2e-682f0577cffc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:04.885074 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f8a094-e094-40b8-8a2e-682f0577cffc None admin] REQUEST_METHOD: `PUT` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:04.885465 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f8a094-e094-40b8-8a2e-682f0577cffc None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:04.885763 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f8a094-e094-40b8-8a2e-682f0577cffc None admin] PATH_INFO: `/v3/projects/b4ae9d03e35549cfbadf1d1001837bb6/groups/73043087b04d4917b9fdcbd262ef56e0/roles/bebc77e8dfd54dbba7079f93218e7c7d` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:04.887984 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f8a094-e094-40b8-8a2e-682f0577cffc None admin] RBAC: Authorizing `identity:create_grant(project_id=b4ae9d03e35549cfbadf1d1001837bb6, group_id=73043087b04d4917b9fdcbd262ef56e0, role_id=bebc77e8dfd54dbba7079f93218e7c7d)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:04.890195 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f8a094-e094-40b8-8a2e-682f0577cffc None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:04.911383 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370b01bd-2ed8-42bd-baff-db841f16fa4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:04.912191 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370b01bd-2ed8-42bd-baff-db841f16fa4f None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:04.912566 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370b01bd-2ed8-42bd-baff-db841f16fa4f None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:04.912984 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370b01bd-2ed8-42bd-baff-db841f16fa4f None admin] PATH_INFO: `/v3/roles` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:04.917450 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370b01bd-2ed8-42bd-baff-db841f16fa4f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:04:04.919181 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370b01bd-2ed8-42bd-baff-db841f16fa4f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:04.920931 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370b01bd-2ed8-42bd-baff-db841f16fa4f None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:04.940210 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 217/411] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Aug 31 01:04:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 346 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 01:04:04.945397 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370b01bd-2ed8-42bd-baff-db841f16fa4f None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:04.963276 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 195/412] 10.222.0.20 () {58 vars in 1442 bytes} [Wed Aug 31 01:04:04 2022] PUT /identity/v3/projects/b4ae9d03e35549cfbadf1d1001837bb6/groups/73043087b04d4917b9fdcbd262ef56e0/roles/bebc77e8dfd54dbba7079f93218e7c7d => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:04:05.118227 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370b01bd-2ed8-42bd-baff-db841f16fa4f None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:05.142158 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:05.176079 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675500a6-0a72-4561-8c90-8ff8c6b54f2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:05.176739 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675500a6-0a72-4561-8c90-8ff8c6b54f2a None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:05.177029 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675500a6-0a72-4561-8c90-8ff8c6b54f2a None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:05.177359 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675500a6-0a72-4561-8c90-8ff8c6b54f2a None admin] PATH_INFO: `/v3/groups/10f6285d94084e98940bda76f4eaf253` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:05.178490 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675500a6-0a72-4561-8c90-8ff8c6b54f2a None admin] RBAC: Authorizing `identity:get_group(group_id=10f6285d94084e98940bda76f4eaf253)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:05.181404 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675500a6-0a72-4561-8c90-8ff8c6b54f2a None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:05.183633 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 218/413] 10.222.0.20 () {56 vars in 1180 bytes} [Wed Aug 31 01:04:04 2022] GET /identity/v3/groups/10f6285d94084e98940bda76f4eaf253 => generated 229 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:04:05.189537 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f8a094-e094-40b8-8a2e-682f0577cffc None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:05.388635 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f8a094-e094-40b8-8a2e-682f0577cffc None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:05.455058 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2a915b-1c64-421e-be20-3425dceec914 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:05.455719 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2a915b-1c64-421e-be20-3425dceec914 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:05.455996 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2a915b-1c64-421e-be20-3425dceec914 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:05.456268 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2a915b-1c64-421e-be20-3425dceec914 None admin] PATH_INFO: `/v3/projects/77c8e2846a11431eb786ad3a5192ea92` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:05.462031 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2a915b-1c64-421e-be20-3425dceec914 None admin] RBAC: Authorizing `identity:get_project(project_id=77c8e2846a11431eb786ad3a5192ea92)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:05.463509 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2a915b-1c64-421e-be20-3425dceec914 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:05.466386 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 196/414] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Aug 31 01:04:05 2022] GET /identity/v3/projects/77c8e2846a11431eb786ad3a5192ea92 => generated 344 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:04:05.474125 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584ce4d7-3c08-4408-a2fe-f97d5d1bd89e None None] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:05.474635 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584ce4d7-3c08-4408-a2fe-f97d5d1bd89e None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:05.476251 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2a915b-1c64-421e-be20-3425dceec914 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:05.476864 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584ce4d7-3c08-4408-a2fe-f97d5d1bd89e None None] PATH_INFO: `/` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:05.482089 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 219/415] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:04:05 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:04:05.490518 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cfa19a-7302-4644-9787-a31eaa3beb3b None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:05.490804 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cfa19a-7302-4644-9787-a31eaa3beb3b None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:05.491084 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cfa19a-7302-4644-9787-a31eaa3beb3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:05.495544 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:05.525777 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-e2cfa19a-7302-4644-9787-a31eaa3beb3b None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:04:05.526994 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-e2cfa19a-7302-4644-9787-a31eaa3beb3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:04:05.617497 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2a915b-1c64-421e-be20-3425dceec914 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:05.660640 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c30a23-d538-4773-8b60-0cd2287c894a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:05.661284 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c30a23-d538-4773-8b60-0cd2287c894a None admin] REQUEST_METHOD: `PUT` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:05.661568 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c30a23-d538-4773-8b60-0cd2287c894a None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:05.661846 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c30a23-d538-4773-8b60-0cd2287c894a None admin] PATH_INFO: `/v3/projects/77c8e2846a11431eb786ad3a5192ea92/groups/10f6285d94084e98940bda76f4eaf253/roles/d009411496b8452b88e77ffb8db015a7` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:05.666597 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c30a23-d538-4773-8b60-0cd2287c894a None admin] RBAC: Authorizing `identity:create_grant(project_id=77c8e2846a11431eb786ad3a5192ea92, group_id=10f6285d94084e98940bda76f4eaf253, role_id=d009411496b8452b88e77ffb8db015a7)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:05.668114 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c30a23-d538-4773-8b60-0cd2287c894a None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:05.682976 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 220/416] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:04:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 01:04:05.736025 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 197/417] 10.222.0.20 () {58 vars in 1442 bytes} [Wed Aug 31 01:04:05 2022] PUT /identity/v3/projects/77c8e2846a11431eb786ad3a5192ea92/groups/10f6285d94084e98940bda76f4eaf253/roles/d009411496b8452b88e77ffb8db015a7 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:04:05.954615 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e275d9-11e7-4757-ba2f-00624bad0a25 None None] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:05.954940 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e275d9-11e7-4757-ba2f-00624bad0a25 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:05.955173 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e275d9-11e7-4757-ba2f-00624bad0a25 None None] PATH_INFO: `/` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:05.956655 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 221/418] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:04:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:04:05.962585 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877d8cc4-704c-422c-a0a9-e67304ffb6ed None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:05.963179 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877d8cc4-704c-422c-a0a9-e67304ffb6ed None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:05.963657 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877d8cc4-704c-422c-a0a9-e67304ffb6ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:05.974626 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e275d9-11e7-4757-ba2f-00624bad0a25 None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:05.984771 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-877d8cc4-704c-422c-a0a9-e67304ffb6ed None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:04:05.986232 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-877d8cc4-704c-422c-a0a9-e67304ffb6ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:04:06.062466 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 198/419] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:04:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 01:04:06.105608 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e275d9-11e7-4757-ba2f-00624bad0a25 None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:06.119170 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:06.150864 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593fb88a-bf3a-48cd-80ca-32ebbb77c778 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:06.151560 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593fb88a-bf3a-48cd-80ca-32ebbb77c778 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:06.151844 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593fb88a-bf3a-48cd-80ca-32ebbb77c778 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:06.152118 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593fb88a-bf3a-48cd-80ca-32ebbb77c778 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:06.165627 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593fb88a-bf3a-48cd-80ca-32ebbb77c778 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 01:04:06.170556 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application [None req-593fb88a-bf3a-48cd-80ca-32ebbb77c778 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 01:04:06.170556 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:04:06.170556 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:04:06.170556 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:04:06.170556 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:04:06.170556 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:04:06.170556 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:04:06.170556 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:04:06.170556 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:04:06.170556 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:04:06.170556 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:04:06.170556 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:04:06.170556 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 01:04:06.170556 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 01:04:06.170556 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 01:04:06.170556 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 01:04:06.171428 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 01:04:06.171428 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 01:04:06.171428 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 01:04:06.171428 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 01:04:06.171428 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:04:06.171428 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:04:06.171428 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:04:06.171428 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:04:06.171428 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:04:06.171428 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:04:06.171428 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:04:06.171428 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application with Lock( Aug 31 01:04:06.171428 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:04:06.171428 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:04:06.171428 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:04:06.171428 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:04:06.172248 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:04:06.172248 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:04:06.172248 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:04:06.172248 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:04:06.172248 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 01:04:06.172248 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 01:04:06.172248 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 01:04:06.172248 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 01:04:06.172248 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 01:04:06.172248 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 01:04:06.172248 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 01:04:06.172248 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Aug 31 01:04:06.172248 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 222/420] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Aug 31 01:04:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:04:06.177619 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877d8cc4-704c-422c-a0a9-e67304ffb6ed None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:06.184745 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:06.319194 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877d8cc4-704c-422c-a0a9-e67304ffb6ed None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:06.367435 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dc28db-8aa9-4f34-a2a6-a9a563a76043 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:06.368820 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dc28db-8aa9-4f34-a2a6-a9a563a76043 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:06.369319 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dc28db-8aa9-4f34-a2a6-a9a563a76043 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:06.369838 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dc28db-8aa9-4f34-a2a6-a9a563a76043 None admin] PATH_INFO: `/v3/roles` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:06.370722 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16dc28db-8aa9-4f34-a2a6-a9a563a76043 None admin] RBAC: Adding query filter params (name=member) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:04:06.371267 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16dc28db-8aa9-4f34-a2a6-a9a563a76043 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:06.372917 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16dc28db-8aa9-4f34-a2a6-a9a563a76043 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:06.385009 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 199/421] 10.222.0.20 () {56 vars in 1101 bytes} [Wed Aug 31 01:04:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:04:06.390221 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593fb88a-bf3a-48cd-80ca-32ebbb77c778 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:06.399811 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dc28db-8aa9-4f34-a2a6-a9a563a76043 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:06.478777 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:06.522206 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dc28db-8aa9-4f34-a2a6-a9a563a76043 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:06.567977 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21503e2e-be35-4e79-9bc9-28a4e6e2fb45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:06.568617 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21503e2e-be35-4e79-9bc9-28a4e6e2fb45 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:06.568919 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21503e2e-be35-4e79-9bc9-28a4e6e2fb45 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:06.569187 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21503e2e-be35-4e79-9bc9-28a4e6e2fb45 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:06.589460 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21503e2e-be35-4e79-9bc9-28a4e6e2fb45 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 01:04:06.594734 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application [None req-21503e2e-be35-4e79-9bc9-28a4e6e2fb45 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 01:04:06.594734 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:04:06.594734 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:04:06.594734 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:04:06.594734 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:04:06.594734 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:04:06.594734 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:04:06.594734 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:04:06.594734 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:04:06.594734 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:04:06.594734 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:04:06.594734 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:04:06.594734 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 01:04:06.594734 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 01:04:06.594734 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 01:04:06.594734 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 01:04:06.595592 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 01:04:06.595592 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 01:04:06.595592 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 01:04:06.595592 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 01:04:06.595592 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:04:06.595592 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:04:06.595592 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:04:06.595592 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:04:06.595592 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:04:06.595592 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:04:06.595592 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:04:06.595592 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application with Lock( Aug 31 01:04:06.595592 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:04:06.595592 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:04:06.595592 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:04:06.595592 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:04:06.596388 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:04:06.596388 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:04:06.596388 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:04:06.596388 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:04:06.596388 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 01:04:06.596388 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 01:04:06.596388 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 01:04:06.596388 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 01:04:06.596388 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 01:04:06.596388 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 01:04:06.596388 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 01:04:06.596388 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Aug 31 01:04:06.604669 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 200/422] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Aug 31 01:04:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 01:04:06.605544 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21503e2e-be35-4e79-9bc9-28a4e6e2fb45 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:06.620500 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593fb88a-bf3a-48cd-80ca-32ebbb77c778 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:06.677837 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87c0c21-cc4b-4b33-a2ca-cd73d5e3254a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:06.678445 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87c0c21-cc4b-4b33-a2ca-cd73d5e3254a None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:06.678724 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87c0c21-cc4b-4b33-a2ca-cd73d5e3254a None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:06.678994 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87c0c21-cc4b-4b33-a2ca-cd73d5e3254a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:06.729215 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:06.766947 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87c0c21-cc4b-4b33-a2ca-cd73d5e3254a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:06.769349 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87c0c21-cc4b-4b33-a2ca-cd73d5e3254a None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:06.769479 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21503e2e-be35-4e79-9bc9-28a4e6e2fb45 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:06.795101 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:06.806680 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2aa5c80-c8a3-4e85-bb5b-b00ee2a097e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:06.808478 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2aa5c80-c8a3-4e85-bb5b-b00ee2a097e2 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:06.808478 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2aa5c80-c8a3-4e85-bb5b-b00ee2a097e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:06.808478 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2aa5c80-c8a3-4e85-bb5b-b00ee2a097e2 None admin] PATH_INFO: `/v3/roles` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:06.808696 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2aa5c80-c8a3-4e85-bb5b-b00ee2a097e2 None admin] RBAC: Adding query filter params (name=member) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:04:06.809009 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2aa5c80-c8a3-4e85-bb5b-b00ee2a097e2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:06.810593 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2aa5c80-c8a3-4e85-bb5b-b00ee2a097e2 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:06.824640 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 201/423] 10.222.0.20 () {56 vars in 1101 bytes} [Wed Aug 31 01:04:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:04:06.826040 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfea522b-ae65-4608-bccd-e08089c08fd6 None None] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:06.826424 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfea522b-ae65-4608-bccd-e08089c08fd6 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:06.826708 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfea522b-ae65-4608-bccd-e08089c08fd6 None None] PATH_INFO: `/` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:06.828295 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 202/424] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:04:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:04:06.829525 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfea522b-ae65-4608-bccd-e08089c08fd6 None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:06.891560 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 223/425] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Aug 31 01:04:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 501 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 01:04:06.895738 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca14f407-a03a-438f-b059-099a7cac2c38 None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:06.895738 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca14f407-a03a-438f-b059-099a7cac2c38 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:06.895738 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca14f407-a03a-438f-b059-099a7cac2c38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:06.927172 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-ca14f407-a03a-438f-b059-099a7cac2c38 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:04:06.929348 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-ca14f407-a03a-438f-b059-099a7cac2c38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:04:06.949897 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfea522b-ae65-4608-bccd-e08089c08fd6 None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:06.997515 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba85115-b16a-4524-b5c4-96b43046d22b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:06.998033 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:06.998563 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba85115-b16a-4524-b5c4-96b43046d22b None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:06.999127 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba85115-b16a-4524-b5c4-96b43046d22b None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:06.999542 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba85115-b16a-4524-b5c4-96b43046d22b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:07.004946 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 224/426] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:04:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 01:04:07.005889 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca14f407-a03a-438f-b059-099a7cac2c38 None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:07.070438 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:07.087998 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba85115-b16a-4524-b5c4-96b43046d22b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:07.089778 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba85115-b16a-4524-b5c4-96b43046d22b None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:07.112561 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca14f407-a03a-438f-b059-099a7cac2c38 None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:07.156081 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d090a18-7d25-4fb3-8974-60cd1c89906d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:07.156632 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d090a18-7d25-4fb3-8974-60cd1c89906d None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:07.156935 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d090a18-7d25-4fb3-8974-60cd1c89906d None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:07.157208 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d090a18-7d25-4fb3-8974-60cd1c89906d None admin] PATH_INFO: `/v3/projects/b4ae9d03e35549cfbadf1d1001837bb6` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:07.158446 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d090a18-7d25-4fb3-8974-60cd1c89906d None admin] RBAC: Authorizing `identity:get_project(project_id=b4ae9d03e35549cfbadf1d1001837bb6)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:07.160501 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d090a18-7d25-4fb3-8974-60cd1c89906d None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:07.162604 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 225/427] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Aug 31 01:04:07 2022] GET /identity/v3/projects/b4ae9d03e35549cfbadf1d1001837bb6 => generated 302 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:04:07.168041 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d090a18-7d25-4fb3-8974-60cd1c89906d None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:07.207705 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 203/428] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Aug 31 01:04:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 01:04:07.215119 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba85115-b16a-4524-b5c4-96b43046d22b None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:07.235674 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:07.270361 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d090a18-7d25-4fb3-8974-60cd1c89906d None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:07.304814 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea0c5a1-03d4-4cba-97aa-08aa751f7558 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:07.305452 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea0c5a1-03d4-4cba-97aa-08aa751f7558 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:07.305730 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea0c5a1-03d4-4cba-97aa-08aa751f7558 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:07.305998 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea0c5a1-03d4-4cba-97aa-08aa751f7558 None admin] PATH_INFO: `/v3/groups/73043087b04d4917b9fdcbd262ef56e0` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:07.307100 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea0c5a1-03d4-4cba-97aa-08aa751f7558 None admin] RBAC: Authorizing `identity:get_group(group_id=73043087b04d4917b9fdcbd262ef56e0)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:07.308527 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea0c5a1-03d4-4cba-97aa-08aa751f7558 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:07.310397 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 226/429] 10.222.0.20 () {56 vars in 1180 bytes} [Wed Aug 31 01:04:07 2022] GET /identity/v3/groups/73043087b04d4917b9fdcbd262ef56e0 => generated 226 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 01:04:07.312001 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:07.315991 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea0c5a1-03d4-4cba-97aa-08aa751f7558 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:07.330058 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba85115-b16a-4524-b5c4-96b43046d22b None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:07.367910 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41779e23-5bc1-4e80-8ccd-fe3e301ec584 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:07.369011 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41779e23-5bc1-4e80-8ccd-fe3e301ec584 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:07.369510 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41779e23-5bc1-4e80-8ccd-fe3e301ec584 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:07.370058 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41779e23-5bc1-4e80-8ccd-fe3e301ec584 None admin] PATH_INFO: `/v3/projects/82238b119c5840de9fc817856cec36fb` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:07.371482 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41779e23-5bc1-4e80-8ccd-fe3e301ec584 None admin] RBAC: Authorizing `identity:get_project(project_id=82238b119c5840de9fc817856cec36fb)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:07.373399 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41779e23-5bc1-4e80-8ccd-fe3e301ec584 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:07.375768 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 204/430] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Aug 31 01:04:07 2022] GET /identity/v3/projects/82238b119c5840de9fc817856cec36fb => generated 298 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:04:07.381489 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41779e23-5bc1-4e80-8ccd-fe3e301ec584 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:07.415913 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea0c5a1-03d4-4cba-97aa-08aa751f7558 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:07.450096 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9595ee37-0395-4041-b906-2bb835633764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:07.450699 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9595ee37-0395-4041-b906-2bb835633764 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:07.450979 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9595ee37-0395-4041-b906-2bb835633764 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:07.451251 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9595ee37-0395-4041-b906-2bb835633764 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:07.451967 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9595ee37-0395-4041-b906-2bb835633764 None admin] RBAC: Adding query filter params (group.id=73043087b04d4917b9fdcbd262ef56e0, role.id=d0b9768f4d014766b9326387f2e11f34, scope.project.id=b4ae9d03e35549cfbadf1d1001837bb6) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:04:07.452275 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9595ee37-0395-4041-b906-2bb835633764 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:07.453725 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9595ee37-0395-4041-b906-2bb835633764 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:07.455457 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:07.459871 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 227/431] 10.222.0.20 () {56 vars in 1376 bytes} [Wed Aug 31 01:04:07 2022] GET /identity/v3/role_assignments?group.id=73043087b04d4917b9fdcbd262ef56e0&scope.project.id=b4ae9d03e35549cfbadf1d1001837bb6&role.id=d0b9768f4d014766b9326387f2e11f34 => generated 611 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 01:04:07.495137 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41779e23-5bc1-4e80-8ccd-fe3e301ec584 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:07.534493 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbde0f84-4548-4403-ad25-539725c21a41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:07.535632 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbde0f84-4548-4403-ad25-539725c21a41 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:07.536114 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbde0f84-4548-4403-ad25-539725c21a41 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:07.536585 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbde0f84-4548-4403-ad25-539725c21a41 None admin] PATH_INFO: `/v3/groups/73043087b04d4917b9fdcbd262ef56e0` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:07.538052 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbde0f84-4548-4403-ad25-539725c21a41 None admin] RBAC: Authorizing `identity:get_group(group_id=73043087b04d4917b9fdcbd262ef56e0)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:07.539826 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbde0f84-4548-4403-ad25-539725c21a41 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:07.542256 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 205/432] 10.222.0.20 () {56 vars in 1180 bytes} [Wed Aug 31 01:04:07 2022] GET /identity/v3/groups/73043087b04d4917b9fdcbd262ef56e0 => generated 226 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 01:04:07.548422 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbde0f84-4548-4403-ad25-539725c21a41 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:07.548575 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9595ee37-0395-4041-b906-2bb835633764 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:07.576743 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:07.664036 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbde0f84-4548-4403-ad25-539725c21a41 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:07.703323 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9595ee37-0395-4041-b906-2bb835633764 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:07.706411 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef282552-e2ee-468e-9e6f-4e0982c659b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:07.707066 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef282552-e2ee-468e-9e6f-4e0982c659b5 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:07.707347 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef282552-e2ee-468e-9e6f-4e0982c659b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:07.707621 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef282552-e2ee-468e-9e6f-4e0982c659b5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:07.720246 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef282552-e2ee-468e-9e6f-4e0982c659b5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 01:04:07.725419 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application [None req-ef282552-e2ee-468e-9e6f-4e0982c659b5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 01:04:07.725419 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:04:07.725419 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:04:07.725419 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:04:07.725419 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:04:07.725419 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:04:07.725419 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:04:07.725419 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:04:07.725419 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:04:07.725419 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:04:07.725419 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:04:07.725419 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:04:07.725419 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 01:04:07.725419 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 01:04:07.725419 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 01:04:07.725419 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 01:04:07.726307 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 01:04:07.726307 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 01:04:07.726307 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 01:04:07.726307 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 01:04:07.726307 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:04:07.726307 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:04:07.726307 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:04:07.726307 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:04:07.726307 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:04:07.726307 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:04:07.726307 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:04:07.726307 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application with Lock( Aug 31 01:04:07.726307 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:04:07.726307 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:04:07.726307 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:04:07.726307 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:04:07.727143 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:04:07.727143 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:04:07.727143 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:04:07.727143 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:04:07.727143 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 01:04:07.727143 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 01:04:07.727143 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 01:04:07.727143 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 01:04:07.727143 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 01:04:07.727143 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 01:04:07.727143 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 01:04:07.727143 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Aug 31 01:04:07.727143 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 206/433] 10.222.0.20 () {56 vars in 1114 bytes} [Wed Aug 31 01:04:07 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:04:07.728339 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6472f6a-59cb-4117-8d6f-dd8a60732aa2 None None] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:07.728339 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6472f6a-59cb-4117-8d6f-dd8a60732aa2 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:07.728339 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6472f6a-59cb-4117-8d6f-dd8a60732aa2 None None] PATH_INFO: `/` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:07.729781 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 207/434] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:04:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:04:07.734986 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc521639-1376-4a4c-b360-ba97fde04e11 None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:07.735275 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc521639-1376-4a4c-b360-ba97fde04e11 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:07.735556 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc521639-1376-4a4c-b360-ba97fde04e11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:07.742528 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0ef024-4012-43ef-b5ab-43614fcb1722 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:07.743805 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0ef024-4012-43ef-b5ab-43614fcb1722 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:07.744110 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0ef024-4012-43ef-b5ab-43614fcb1722 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:07.744451 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0ef024-4012-43ef-b5ab-43614fcb1722 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:07.745219 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0ef024-4012-43ef-b5ab-43614fcb1722 None admin] RBAC: Adding query filter params (group.id=73043087b04d4917b9fdcbd262ef56e0, role.id=d0b9768f4d014766b9326387f2e11f34, scope.project.id=82238b119c5840de9fc817856cec36fb) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:04:07.745537 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0ef024-4012-43ef-b5ab-43614fcb1722 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:07.746988 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0ef024-4012-43ef-b5ab-43614fcb1722 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:07.754992 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 228/435] 10.222.0.20 () {56 vars in 1376 bytes} [Wed Aug 31 01:04:07 2022] GET /identity/v3/role_assignments?group.id=73043087b04d4917b9fdcbd262ef56e0&scope.project.id=82238b119c5840de9fc817856cec36fb&role.id=d0b9768f4d014766b9326387f2e11f34 => generated 611 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:04:07.755836 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0ef024-4012-43ef-b5ab-43614fcb1722 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:07.757807 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-bc521639-1376-4a4c-b360-ba97fde04e11 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:04:07.761252 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-bc521639-1376-4a4c-b360-ba97fde04e11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:04:07.843461 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 208/436] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:04:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 01:04:07.844774 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:07.889665 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0ef024-4012-43ef-b5ab-43614fcb1722 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:07.930129 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8184576-2fee-4dae-b021-e230c30f4834 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:07.931273 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8184576-2fee-4dae-b021-e230c30f4834 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:07.931524 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8184576-2fee-4dae-b021-e230c30f4834 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:07.931791 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8184576-2fee-4dae-b021-e230c30f4834 None admin] PATH_INFO: `/v3/roles` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:07.932447 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8184576-2fee-4dae-b021-e230c30f4834 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:04:07.932779 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8184576-2fee-4dae-b021-e230c30f4834 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:07.934148 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8184576-2fee-4dae-b021-e230c30f4834 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:07.942608 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 229/437] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Aug 31 01:04:07 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:04:07.948141 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc521639-1376-4a4c-b360-ba97fde04e11 None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:07.965148 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:08.058701 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc521639-1376-4a4c-b360-ba97fde04e11 None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:08.095603 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904ef3ce-136b-4684-8066-87d1a6801dff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:08.096455 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904ef3ce-136b-4684-8066-87d1a6801dff None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:08.096916 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904ef3ce-136b-4684-8066-87d1a6801dff None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:08.097338 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904ef3ce-136b-4684-8066-87d1a6801dff None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:08.117715 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8184576-2fee-4dae-b021-e230c30f4834 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:08.180331 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904ef3ce-136b-4684-8066-87d1a6801dff None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:08.182116 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904ef3ce-136b-4684-8066-87d1a6801dff None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:08.197141 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:08.233415 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8184576-2fee-4dae-b021-e230c30f4834 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:08.255799 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:08.266457 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9874918c-5601-41b7-a9f0-1aa4a274525c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:08.267077 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9874918c-5601-41b7-a9f0-1aa4a274525c None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:08.267360 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9874918c-5601-41b7-a9f0-1aa4a274525c None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:08.267629 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9874918c-5601-41b7-a9f0-1aa4a274525c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:08.277542 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9874918c-5601-41b7-a9f0-1aa4a274525c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 01:04:08.282343 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application [None req-9874918c-5601-41b7-a9f0-1aa4a274525c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 01:04:08.282343 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:04:08.282343 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:04:08.282343 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:04:08.282343 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:04:08.282343 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:04:08.282343 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:04:08.282343 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:04:08.282343 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:04:08.282343 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:04:08.282343 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:04:08.282343 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:04:08.282343 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 01:04:08.282343 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 01:04:08.282343 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 01:04:08.282343 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 01:04:08.283272 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 01:04:08.283272 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 01:04:08.283272 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 01:04:08.283272 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 01:04:08.283272 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:04:08.283272 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:04:08.283272 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:04:08.283272 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:04:08.283272 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:04:08.283272 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:04:08.283272 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:04:08.283272 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application with Lock( Aug 31 01:04:08.283272 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:04:08.283272 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:04:08.283272 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:04:08.283272 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:04:08.289230 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:04:08.289230 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:04:08.289230 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:04:08.289230 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:04:08.289230 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 01:04:08.289230 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 01:04:08.289230 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 01:04:08.289230 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 01:04:08.289230 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 01:04:08.289230 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 01:04:08.289230 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 01:04:08.289230 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Aug 31 01:04:08.289230 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 230/438] 10.222.0.20 () {56 vars in 1114 bytes} [Wed Aug 31 01:04:08 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 01:04:08.289230 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b0f621-2bcd-45e3-8403-7793a57ea75e None None] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:08.290050 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b0f621-2bcd-45e3-8403-7793a57ea75e None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:08.290050 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b0f621-2bcd-45e3-8403-7793a57ea75e None None] PATH_INFO: `/` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:08.290167 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 209/439] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Aug 31 01:04:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 01:04:08.290243 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 231/440] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:04:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:04:08.290307 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904ef3ce-136b-4684-8066-87d1a6801dff None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:08.293757 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3039e27d-10ca-46e2-966a-12aa3e0d3947 None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:08.294061 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3039e27d-10ca-46e2-966a-12aa3e0d3947 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:08.294330 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3039e27d-10ca-46e2-966a-12aa3e0d3947 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:08.311739 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-3039e27d-10ca-46e2-966a-12aa3e0d3947 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:04:08.312939 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-3039e27d-10ca-46e2-966a-12aa3e0d3947 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:04:08.378124 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 232/441] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:04:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 01:04:08.379097 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3039e27d-10ca-46e2-966a-12aa3e0d3947 None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:08.388263 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904ef3ce-136b-4684-8066-87d1a6801dff None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:08.403608 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:08.425000 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0bac82-b027-4bf9-8ac1-6d31de6ca9d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:08.425912 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0bac82-b027-4bf9-8ac1-6d31de6ca9d4 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:08.426192 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0bac82-b027-4bf9-8ac1-6d31de6ca9d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:08.426465 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0bac82-b027-4bf9-8ac1-6d31de6ca9d4 None admin] PATH_INFO: `/v3/roles` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:08.427126 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0bac82-b027-4bf9-8ac1-6d31de6ca9d4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:04:08.427426 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0bac82-b027-4bf9-8ac1-6d31de6ca9d4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:08.428813 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0bac82-b027-4bf9-8ac1-6d31de6ca9d4 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:08.438680 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 210/442] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Aug 31 01:04:08 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:04:08.443926 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0bac82-b027-4bf9-8ac1-6d31de6ca9d4 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:08.493735 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3039e27d-10ca-46e2-966a-12aa3e0d3947 None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:08.509701 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:08.530310 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d98996-0def-4ac2-8b63-53c9bd3d0d56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:08.530939 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d98996-0def-4ac2-8b63-53c9bd3d0d56 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:08.531224 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d98996-0def-4ac2-8b63-53c9bd3d0d56 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:08.531498 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d98996-0def-4ac2-8b63-53c9bd3d0d56 None admin] PATH_INFO: `/v3/projects/82238b119c5840de9fc817856cec36fb` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:08.532622 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d98996-0def-4ac2-8b63-53c9bd3d0d56 None admin] RBAC: Authorizing `identity:get_project(project_id=82238b119c5840de9fc817856cec36fb)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:08.534086 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d98996-0def-4ac2-8b63-53c9bd3d0d56 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:08.535703 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 233/443] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Aug 31 01:04:08 2022] GET /identity/v3/projects/82238b119c5840de9fc817856cec36fb => generated 298 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:04:08.541881 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d98996-0def-4ac2-8b63-53c9bd3d0d56 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:08.551442 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0bac82-b027-4bf9-8ac1-6d31de6ca9d4 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:08.588330 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a05afae-a6b3-4470-b5ac-19dc254a480d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:08.588968 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a05afae-a6b3-4470-b5ac-19dc254a480d None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:08.589257 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a05afae-a6b3-4470-b5ac-19dc254a480d None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:08.589533 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a05afae-a6b3-4470-b5ac-19dc254a480d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:08.631568 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:08.648326 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d98996-0def-4ac2-8b63-53c9bd3d0d56 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:08.656970 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a05afae-a6b3-4470-b5ac-19dc254a480d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:08.658450 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a05afae-a6b3-4470-b5ac-19dc254a480d None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:08.682282 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24a783d-44dd-4c0b-aafd-ca622a1edea6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:08.682904 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24a783d-44dd-4c0b-aafd-ca622a1edea6 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:08.683188 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24a783d-44dd-4c0b-aafd-ca622a1edea6 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:08.683476 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24a783d-44dd-4c0b-aafd-ca622a1edea6 None admin] PATH_INFO: `/v3/groups/73043087b04d4917b9fdcbd262ef56e0` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:08.684593 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24a783d-44dd-4c0b-aafd-ca622a1edea6 None admin] RBAC: Authorizing `identity:get_group(group_id=73043087b04d4917b9fdcbd262ef56e0)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:08.686050 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24a783d-44dd-4c0b-aafd-ca622a1edea6 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:08.687604 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 234/444] 10.222.0.20 () {56 vars in 1180 bytes} [Wed Aug 31 01:04:08 2022] GET /identity/v3/groups/73043087b04d4917b9fdcbd262ef56e0 => generated 226 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 01:04:08.688358 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24a783d-44dd-4c0b-aafd-ca622a1edea6 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:08.736429 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:08.761077 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 211/445] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Aug 31 01:04:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 31 01:04:08.761848 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a05afae-a6b3-4470-b5ac-19dc254a480d None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:08.791146 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24a783d-44dd-4c0b-aafd-ca622a1edea6 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:08.823800 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2546f237-1f03-4b62-a988-cdc199af0395 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:08.824686 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2546f237-1f03-4b62-a988-cdc199af0395 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:08.824993 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2546f237-1f03-4b62-a988-cdc199af0395 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:08.825272 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2546f237-1f03-4b62-a988-cdc199af0395 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:08.835142 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2546f237-1f03-4b62-a988-cdc199af0395 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 01:04:08.838450 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:08.840708 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application [None req-2546f237-1f03-4b62-a988-cdc199af0395 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 01:04:08.840708 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:04:08.840708 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:04:08.840708 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:04:08.840708 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:04:08.840708 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:04:08.840708 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:04:08.840708 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:04:08.840708 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:04:08.840708 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:04:08.840708 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:04:08.840708 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:04:08.840708 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 01:04:08.840708 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 01:04:08.840708 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 01:04:08.840708 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 01:04:08.841836 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 01:04:08.841836 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 01:04:08.841836 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 01:04:08.841836 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 01:04:08.841836 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:04:08.841836 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:04:08.841836 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:04:08.841836 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:04:08.841836 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:04:08.841836 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:04:08.841836 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:04:08.841836 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application with Lock( Aug 31 01:04:08.841836 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:04:08.841836 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:04:08.841836 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:04:08.841836 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:04:08.842641 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:04:08.842641 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:04:08.842641 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:04:08.842641 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:04:08.842641 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 01:04:08.842641 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 01:04:08.842641 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 01:04:08.842641 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 01:04:08.842641 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 01:04:08.842641 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 01:04:08.842641 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 01:04:08.842641 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Aug 31 01:04:08.842641 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 235/446] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Aug 31 01:04:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:04:08.842641 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2546f237-1f03-4b62-a988-cdc199af0395 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:08.864242 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a05afae-a6b3-4470-b5ac-19dc254a480d None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:08.905717 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83aeb3cb-2ba5-4da3-a7fe-d6747846c53d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:08.906339 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83aeb3cb-2ba5-4da3-a7fe-d6747846c53d None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:08.906624 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83aeb3cb-2ba5-4da3-a7fe-d6747846c53d None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:08.906904 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83aeb3cb-2ba5-4da3-a7fe-d6747846c53d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:08.907638 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83aeb3cb-2ba5-4da3-a7fe-d6747846c53d None admin] RBAC: Adding query filter params (group.id=73043087b04d4917b9fdcbd262ef56e0, role.id=bebc77e8dfd54dbba7079f93218e7c7d, scope.project.id=82238b119c5840de9fc817856cec36fb) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:04:08.907952 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83aeb3cb-2ba5-4da3-a7fe-d6747846c53d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:08.909449 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83aeb3cb-2ba5-4da3-a7fe-d6747846c53d None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:08.918744 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 212/447] 10.222.0.20 () {56 vars in 1376 bytes} [Wed Aug 31 01:04:08 2022] GET /identity/v3/role_assignments?group.id=73043087b04d4917b9fdcbd262ef56e0&scope.project.id=82238b119c5840de9fc817856cec36fb&role.id=bebc77e8dfd54dbba7079f93218e7c7d => generated 611 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:04:08.919875 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83aeb3cb-2ba5-4da3-a7fe-d6747846c53d None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:08.947019 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2546f237-1f03-4b62-a988-cdc199af0395 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:08.960119 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:08.988363 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4518a51-61b4-4cfe-b86e-7ec3857be561 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:08.989168 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4518a51-61b4-4cfe-b86e-7ec3857be561 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:08.989881 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4518a51-61b4-4cfe-b86e-7ec3857be561 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:08.990515 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4518a51-61b4-4cfe-b86e-7ec3857be561 None admin] PATH_INFO: `/v3/projects/b4ae9d03e35549cfbadf1d1001837bb6` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:08.992192 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4518a51-61b4-4cfe-b86e-7ec3857be561 None admin] RBAC: Authorizing `identity:get_project(project_id=b4ae9d03e35549cfbadf1d1001837bb6)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:08.995037 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4518a51-61b4-4cfe-b86e-7ec3857be561 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:08.996985 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 236/448] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Aug 31 01:04:08 2022] GET /identity/v3/projects/b4ae9d03e35549cfbadf1d1001837bb6 => generated 302 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:04:09.002415 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4518a51-61b4-4cfe-b86e-7ec3857be561 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:09.038627 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83aeb3cb-2ba5-4da3-a7fe-d6747846c53d None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:09.079516 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977a8b78-88a5-4b0c-8e24-f8293b041b63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:09.080265 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977a8b78-88a5-4b0c-8e24-f8293b041b63 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:09.080555 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977a8b78-88a5-4b0c-8e24-f8293b041b63 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:09.080885 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977a8b78-88a5-4b0c-8e24-f8293b041b63 None admin] PATH_INFO: `/v3/roles` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:09.081566 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977a8b78-88a5-4b0c-8e24-f8293b041b63 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:04:09.081864 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977a8b78-88a5-4b0c-8e24-f8293b041b63 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:09.083251 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977a8b78-88a5-4b0c-8e24-f8293b041b63 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:09.091656 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:09.096089 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 213/449] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Aug 31 01:04:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:04:09.101761 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977a8b78-88a5-4b0c-8e24-f8293b041b63 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:09.118279 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4518a51-61b4-4cfe-b86e-7ec3857be561 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:09.160384 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40c8d04-6697-4f3e-88aa-f923df20927c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:09.161175 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40c8d04-6697-4f3e-88aa-f923df20927c None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:09.161458 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40c8d04-6697-4f3e-88aa-f923df20927c None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:09.161735 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40c8d04-6697-4f3e-88aa-f923df20927c None admin] PATH_INFO: `/v3/groups/73043087b04d4917b9fdcbd262ef56e0` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:09.162918 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40c8d04-6697-4f3e-88aa-f923df20927c None admin] RBAC: Authorizing `identity:get_group(group_id=73043087b04d4917b9fdcbd262ef56e0)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:09.164364 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40c8d04-6697-4f3e-88aa-f923df20927c None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:09.166678 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 237/450] 10.222.0.20 () {56 vars in 1180 bytes} [Wed Aug 31 01:04:09 2022] GET /identity/v3/groups/73043087b04d4917b9fdcbd262ef56e0 => generated 226 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 01:04:09.172229 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40c8d04-6697-4f3e-88aa-f923df20927c None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:09.220099 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977a8b78-88a5-4b0c-8e24-f8293b041b63 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:09.232659 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:09.258821 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7699d9cb-b16c-4559-b81b-d5b4f2f94df9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:09.259436 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7699d9cb-b16c-4559-b81b-d5b4f2f94df9 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:09.259715 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7699d9cb-b16c-4559-b81b-d5b4f2f94df9 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:09.259984 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7699d9cb-b16c-4559-b81b-d5b4f2f94df9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:09.299712 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40c8d04-6697-4f3e-88aa-f923df20927c None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:09.330035 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:09.332929 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7699d9cb-b16c-4559-b81b-d5b4f2f94df9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:09.334942 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7699d9cb-b16c-4559-b81b-d5b4f2f94df9 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:09.339734 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d8893b-5b51-4dcc-b925-0e59e5c57794 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:09.340525 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d8893b-5b51-4dcc-b925-0e59e5c57794 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:09.344273 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d8893b-5b51-4dcc-b925-0e59e5c57794 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:09.344273 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d8893b-5b51-4dcc-b925-0e59e5c57794 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:09.344273 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d8893b-5b51-4dcc-b925-0e59e5c57794 None admin] RBAC: Adding query filter params (group.id=73043087b04d4917b9fdcbd262ef56e0, role.id=bebc77e8dfd54dbba7079f93218e7c7d, scope.project.id=b4ae9d03e35549cfbadf1d1001837bb6) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:04:09.344273 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d8893b-5b51-4dcc-b925-0e59e5c57794 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:09.344273 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d8893b-5b51-4dcc-b925-0e59e5c57794 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:09.350033 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 238/451] 10.222.0.20 () {56 vars in 1376 bytes} [Wed Aug 31 01:04:09 2022] GET /identity/v3/role_assignments?group.id=73043087b04d4917b9fdcbd262ef56e0&scope.project.id=b4ae9d03e35549cfbadf1d1001837bb6&role.id=bebc77e8dfd54dbba7079f93218e7c7d => generated 611 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 01:04:09.453106 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 214/452] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Aug 31 01:04:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 01:04:09.459716 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d8893b-5b51-4dcc-b925-0e59e5c57794 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:09.560535 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d8893b-5b51-4dcc-b925-0e59e5c57794 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:09.571452 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:09.595444 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af288002-3f08-47f6-b358-aa6ab9a058a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:09.596080 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af288002-3f08-47f6-b358-aa6ab9a058a1 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:09.596368 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af288002-3f08-47f6-b358-aa6ab9a058a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:09.596671 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af288002-3f08-47f6-b358-aa6ab9a058a1 None admin] PATH_INFO: `/v3/projects/77c8e2846a11431eb786ad3a5192ea92` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:09.597840 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af288002-3f08-47f6-b358-aa6ab9a058a1 None admin] RBAC: Authorizing `identity:get_project(project_id=77c8e2846a11431eb786ad3a5192ea92)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:09.599325 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af288002-3f08-47f6-b358-aa6ab9a058a1 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:09.601019 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 239/453] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Aug 31 01:04:09 2022] GET /identity/v3/projects/77c8e2846a11431eb786ad3a5192ea92 => generated 344 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:04:09.606758 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7699d9cb-b16c-4559-b81b-d5b4f2f94df9 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:09.705797 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:09.708617 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7699d9cb-b16c-4559-b81b-d5b4f2f94df9 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:09.741242 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0919c115-80d8-4747-944c-f37cf2137837 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:09.741863 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0919c115-80d8-4747-944c-f37cf2137837 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:09.742140 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0919c115-80d8-4747-944c-f37cf2137837 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:09.742412 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0919c115-80d8-4747-944c-f37cf2137837 None admin] PATH_INFO: `/v3/groups/10f6285d94084e98940bda76f4eaf253` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:09.744032 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0919c115-80d8-4747-944c-f37cf2137837 None admin] RBAC: Authorizing `identity:get_group(group_id=10f6285d94084e98940bda76f4eaf253)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:09.745493 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0919c115-80d8-4747-944c-f37cf2137837 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:09.747039 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 215/454] 10.222.0.20 () {56 vars in 1180 bytes} [Wed Aug 31 01:04:09 2022] GET /identity/v3/groups/10f6285d94084e98940bda76f4eaf253 => generated 229 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:04:09.752986 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af288002-3f08-47f6-b358-aa6ab9a058a1 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:09.848656 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af288002-3f08-47f6-b358-aa6ab9a058a1 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:09.880650 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c597a642-20fc-48a0-9a0c-e51cc8ac1700 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:09.881261 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c597a642-20fc-48a0-9a0c-e51cc8ac1700 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:09.881546 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c597a642-20fc-48a0-9a0c-e51cc8ac1700 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:09.881822 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c597a642-20fc-48a0-9a0c-e51cc8ac1700 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:09.882543 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c597a642-20fc-48a0-9a0c-e51cc8ac1700 None admin] RBAC: Adding query filter params (group.id=10f6285d94084e98940bda76f4eaf253, role.id=d009411496b8452b88e77ffb8db015a7, scope.project.id=77c8e2846a11431eb786ad3a5192ea92) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:04:09.882854 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c597a642-20fc-48a0-9a0c-e51cc8ac1700 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:09.884880 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c597a642-20fc-48a0-9a0c-e51cc8ac1700 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:09.890374 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 240/455] 10.222.0.20 () {56 vars in 1376 bytes} [Wed Aug 31 01:04:09 2022] GET /identity/v3/role_assignments?group.id=10f6285d94084e98940bda76f4eaf253&scope.project.id=77c8e2846a11431eb786ad3a5192ea92&role.id=d009411496b8452b88e77ffb8db015a7 => generated 611 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:04:12.203789 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e11f502-1a68-4723-b856-9a60e17c8815 None None] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:12.204401 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e11f502-1a68-4723-b856-9a60e17c8815 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:12.204401 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e11f502-1a68-4723-b856-9a60e17c8815 None None] PATH_INFO: `/` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:12.205936 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 216/456] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:04:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:04:12.211796 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604a1f08-31d6-442b-bce4-d4a559955f59 None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:12.212193 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604a1f08-31d6-442b-bce4-d4a559955f59 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:12.212356 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604a1f08-31d6-442b-bce4-d4a559955f59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:12.233724 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-604a1f08-31d6-442b-bce4-d4a559955f59 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:04:12.234904 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-604a1f08-31d6-442b-bce4-d4a559955f59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:04:12.282985 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:12.325221 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 241/457] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:04:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 01:04:12.462654 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6566fb2f-d663-4d81-a7ab-e8a6aea3446d None None] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:12.462961 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6566fb2f-d663-4d81-a7ab-e8a6aea3446d None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:12.463142 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6566fb2f-d663-4d81-a7ab-e8a6aea3446d None None] PATH_INFO: `/` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:12.464606 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 217/458] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:04:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:04:12.479783 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da52b3e1-4879-45de-8d70-692170449e94 None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:12.480075 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da52b3e1-4879-45de-8d70-692170449e94 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:12.480348 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da52b3e1-4879-45de-8d70-692170449e94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:12.505706 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-da52b3e1-4879-45de-8d70-692170449e94 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:04:12.506964 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-da52b3e1-4879-45de-8d70-692170449e94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:04:12.514747 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fef7859-431f-4ea4-b439-f7d1176fdf64 None None] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:12.514747 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fef7859-431f-4ea4-b439-f7d1176fdf64 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:12.514747 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fef7859-431f-4ea4-b439-f7d1176fdf64 None None] PATH_INFO: `/` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:12.515374 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 218/459] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:04:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:04:12.521535 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922592b7-0ba1-41e9-bb7c-fae123e11543 None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:12.522066 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922592b7-0ba1-41e9-bb7c-fae123e11543 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:12.522541 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922592b7-0ba1-41e9-bb7c-fae123e11543 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:12.557199 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-922592b7-0ba1-41e9-bb7c-fae123e11543 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:04:12.558794 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-922592b7-0ba1-41e9-bb7c-fae123e11543 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:04:12.580773 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 242/460] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:04:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 01:04:12.672412 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 219/461] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:04:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 01:04:12.684403 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da52b3e1-4879-45de-8d70-692170449e94 None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:12.787571 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da52b3e1-4879-45de-8d70-692170449e94 None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:12.793139 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:12.840145 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1e9e7e-8703-4bcd-9c25-92fcbc86e4fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:12.841182 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1e9e7e-8703-4bcd-9c25-92fcbc86e4fa None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:12.841692 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1e9e7e-8703-4bcd-9c25-92fcbc86e4fa None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:12.842165 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1e9e7e-8703-4bcd-9c25-92fcbc86e4fa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:12.852547 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1e9e7e-8703-4bcd-9c25-92fcbc86e4fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:12.858276 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1e9e7e-8703-4bcd-9c25-92fcbc86e4fa None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:12.873893 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application [None req-4b1e9e7e-8703-4bcd-9c25-92fcbc86e4fa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 01:04:12.873893 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:04:12.873893 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:04:12.873893 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:04:12.873893 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:04:12.873893 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:04:12.873893 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:04:12.873893 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:04:12.873893 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:04:12.873893 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:04:12.873893 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:04:12.873893 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:04:12.873893 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 01:04:12.873893 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 01:04:12.873893 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 01:04:12.873893 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 01:04:12.874818 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:04:12.874818 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:04:12.874818 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:04:12.874818 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:04:12.874818 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:04:12.874818 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:04:12.874818 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:04:12.874818 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application with Lock( Aug 31 01:04:12.874818 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:04:12.874818 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:04:12.874818 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:04:12.874818 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:04:12.874818 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:04:12.874818 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:04:12.874818 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:04:12.874818 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:04:12.874818 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 01:04:12.875652 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 01:04:12.875652 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 01:04:12.875652 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Aug 31 01:04:12.877602 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 243/462] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 01:04:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:04:12.885851 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922592b7-0ba1-41e9-bb7c-fae123e11543 None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:12.965703 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1e9e7e-8703-4bcd-9c25-92fcbc86e4fa None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:13.013534 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:13.018015 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922592b7-0ba1-41e9-bb7c-fae123e11543 None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:13.052709 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4f1c18-c843-4a8f-909e-51d3b858b3d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:13.053284 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4f1c18-c843-4a8f-909e-51d3b858b3d3 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:13.053565 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4f1c18-c843-4a8f-909e-51d3b858b3d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:13.053839 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4f1c18-c843-4a8f-909e-51d3b858b3d3 None admin] PATH_INFO: `/v3/domains` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:13.054481 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4f1c18-c843-4a8f-909e-51d3b858b3d3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:04:13.054782 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4f1c18-c843-4a8f-909e-51d3b858b3d3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:13.056684 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4f1c18-c843-4a8f-909e-51d3b858b3d3 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:13.075437 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 220/463] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 31 01:04:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:04:13.076284 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4f1c18-c843-4a8f-909e-51d3b858b3d3 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:13.111797 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1e9e7e-8703-4bcd-9c25-92fcbc86e4fa None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:13.163619 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78268c5e-341b-407a-a0e2-8c29a54c1278 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:13.165257 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78268c5e-341b-407a-a0e2-8c29a54c1278 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:13.165933 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78268c5e-341b-407a-a0e2-8c29a54c1278 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:13.166216 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78268c5e-341b-407a-a0e2-8c29a54c1278 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:13.181881 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78268c5e-341b-407a-a0e2-8c29a54c1278 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:13.183625 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78268c5e-341b-407a-a0e2-8c29a54c1278 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:13.184363 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4f1c18-c843-4a8f-909e-51d3b858b3d3 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:13.203292 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application [None req-78268c5e-341b-407a-a0e2-8c29a54c1278 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 01:04:13.203292 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:04:13.203292 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:04:13.203292 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:04:13.203292 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:04:13.203292 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:04:13.203292 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:04:13.203292 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:04:13.203292 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:04:13.203292 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:04:13.203292 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:04:13.203292 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:04:13.203292 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 01:04:13.203292 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 01:04:13.203292 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 01:04:13.203292 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 01:04:13.204245 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:04:13.204245 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:04:13.204245 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:04:13.204245 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:04:13.204245 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:04:13.204245 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:04:13.204245 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:04:13.204245 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application with Lock( Aug 31 01:04:13.204245 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:04:13.204245 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:04:13.204245 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:04:13.204245 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:04:13.204245 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:04:13.204245 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:04:13.204245 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:04:13.204245 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:04:13.204245 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 01:04:13.205148 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 01:04:13.205148 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 01:04:13.205148 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Aug 31 01:04:13.205148 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 244/464] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 01:04:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 01:04:13.209323 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78268c5e-341b-407a-a0e2-8c29a54c1278 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:13.222807 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:13.224911 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ca8a1a-a84d-4b3d-9fc1-392ea9c1c872 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:13.226108 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ca8a1a-a84d-4b3d-9fc1-392ea9c1c872 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:13.226642 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ca8a1a-a84d-4b3d-9fc1-392ea9c1c872 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:13.227630 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ca8a1a-a84d-4b3d-9fc1-392ea9c1c872 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:13.239340 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ca8a1a-a84d-4b3d-9fc1-392ea9c1c872 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:13.241145 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ca8a1a-a84d-4b3d-9fc1-392ea9c1c872 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:13.258870 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application [None req-c4ca8a1a-a84d-4b3d-9fc1-392ea9c1c872 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 01:04:13.258870 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:04:13.258870 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:04:13.258870 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:04:13.258870 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:04:13.258870 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:04:13.258870 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:04:13.258870 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:04:13.258870 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:04:13.258870 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:04:13.258870 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:04:13.258870 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:04:13.258870 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 01:04:13.258870 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 01:04:13.258870 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 01:04:13.258870 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 01:04:13.259823 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:04:13.259823 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:04:13.259823 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:04:13.259823 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:04:13.259823 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:04:13.259823 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:04:13.259823 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:04:13.259823 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application with Lock( Aug 31 01:04:13.259823 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:04:13.259823 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:04:13.259823 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:04:13.259823 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:04:13.259823 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:04:13.259823 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:04:13.259823 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:04:13.259823 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:04:13.259823 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 01:04:13.260676 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 01:04:13.260676 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 01:04:13.260676 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Aug 31 01:04:13.260676 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 221/465] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 01:04:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:04:13.260925 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ca8a1a-a84d-4b3d-9fc1-392ea9c1c872 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:13.274782 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:13.333570 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78268c5e-341b-407a-a0e2-8c29a54c1278 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:13.370416 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ee9c2e-52f9-41fd-a650-59318d8866ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:13.371019 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ee9c2e-52f9-41fd-a650-59318d8866ac None admin] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:13.371296 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ee9c2e-52f9-41fd-a650-59318d8866ac None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:13.371571 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ee9c2e-52f9-41fd-a650-59318d8866ac None admin] PATH_INFO: `/v3/users` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:13.372214 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ee9c2e-52f9-41fd-a650-59318d8866ac None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:13.373687 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ee9c2e-52f9-41fd-a650-59318d8866ac None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:13.378822 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ca8a1a-a84d-4b3d-9fc1-392ea9c1c872 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:13.412884 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e516f42c-57cd-4b7d-be66-61b62a0b3e66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:13.413497 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e516f42c-57cd-4b7d-be66-61b62a0b3e66 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:13.413779 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e516f42c-57cd-4b7d-be66-61b62a0b3e66 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:13.414054 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e516f42c-57cd-4b7d-be66-61b62a0b3e66 None admin] PATH_INFO: `/v3/domains` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:13.415207 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e516f42c-57cd-4b7d-be66-61b62a0b3e66 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:04:13.415510 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e516f42c-57cd-4b7d-be66-61b62a0b3e66 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:13.416960 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e516f42c-57cd-4b7d-be66-61b62a0b3e66 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:13.428469 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 222/466] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 31 01:04:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:04:13.429691 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997f998c-990e-4750-b1fc-42bad62ae5af None None] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:13.430644 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997f998c-990e-4750-b1fc-42bad62ae5af None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:13.430929 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997f998c-990e-4750-b1fc-42bad62ae5af None None] PATH_INFO: `/` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:13.434062 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 223/467] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:04:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:04:13.434062 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997f998c-990e-4750-b1fc-42bad62ae5af None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:13.480567 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 245/468] 10.222.0.20 () {60 vars in 1132 bytes} [Wed Aug 31 01:04:13 2022] POST /identity/v3/users => generated 246 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:04:13.481831 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ee9c2e-52f9-41fd-a650-59318d8866ac None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:13.517779 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:13.562105 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997f998c-990e-4750-b1fc-42bad62ae5af None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:13.577549 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:13.611757 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb02d64-1689-4008-a38c-98a515343de9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:13.612987 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb02d64-1689-4008-a38c-98a515343de9 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:13.613241 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb02d64-1689-4008-a38c-98a515343de9 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:13.613497 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb02d64-1689-4008-a38c-98a515343de9 None admin] PATH_INFO: `/v3/domains` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:13.614168 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb02d64-1689-4008-a38c-98a515343de9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:04:13.614481 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb02d64-1689-4008-a38c-98a515343de9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:13.616276 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb02d64-1689-4008-a38c-98a515343de9 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:13.627490 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ee9c2e-52f9-41fd-a650-59318d8866ac None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:13.633427 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 224/469] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 31 01:04:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:04:13.634716 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550b2fdd-5648-4bc6-b0fc-d203615e8d78 None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:13.635056 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550b2fdd-5648-4bc6-b0fc-d203615e8d78 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:13.635342 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550b2fdd-5648-4bc6-b0fc-d203615e8d78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:13.659037 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-550b2fdd-5648-4bc6-b0fc-d203615e8d78 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:04:13.660512 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-550b2fdd-5648-4bc6-b0fc-d203615e8d78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:04:13.681258 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a4c65c-5032-4ff9-b993-b9751f37087c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:13.682219 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a4c65c-5032-4ff9-b993-b9751f37087c None admin] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:13.682219 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a4c65c-5032-4ff9-b993-b9751f37087c None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:13.682717 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a4c65c-5032-4ff9-b993-b9751f37087c None admin] PATH_INFO: `/v3/users` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:13.683074 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a4c65c-5032-4ff9-b993-b9751f37087c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:13.684571 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a4c65c-5032-4ff9-b993-b9751f37087c None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:13.739516 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 225/470] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:04:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 01:04:13.740426 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550b2fdd-5648-4bc6-b0fc-d203615e8d78 None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:13.765695 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 246/471] 10.222.0.20 () {60 vars in 1132 bytes} [Wed Aug 31 01:04:13 2022] POST /identity/v3/users => generated 248 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:04:13.767026 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae7e499-9ef5-4c2d-83d5-3ea438af149a None None] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:13.767341 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae7e499-9ef5-4c2d-83d5-3ea438af149a None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:13.767593 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae7e499-9ef5-4c2d-83d5-3ea438af149a None None] PATH_INFO: `/` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:13.772950 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 247/472] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:04:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:04:13.777535 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a99d30-1c74-441c-840a-1e182600df0b None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:13.778138 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a99d30-1c74-441c-840a-1e182600df0b None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:13.778708 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a99d30-1c74-441c-840a-1e182600df0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:13.819468 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-e6a99d30-1c74-441c-840a-1e182600df0b None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:04:13.820897 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-e6a99d30-1c74-441c-840a-1e182600df0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:04:13.872192 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:13.933558 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 248/473] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:04:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 01:04:13.947882 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550b2fdd-5648-4bc6-b0fc-d203615e8d78 None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:13.984428 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572c72b1-6119-4743-8dab-f6bafc5e3aec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:13.985080 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572c72b1-6119-4743-8dab-f6bafc5e3aec None admin] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:13.985372 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572c72b1-6119-4743-8dab-f6bafc5e3aec None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:13.985656 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572c72b1-6119-4743-8dab-f6bafc5e3aec None admin] PATH_INFO: `/v3/users` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:13.986301 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572c72b1-6119-4743-8dab-f6bafc5e3aec None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:13.987814 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572c72b1-6119-4743-8dab-f6bafc5e3aec None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:14.067658 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a99d30-1c74-441c-840a-1e182600df0b None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:14.083589 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 226/474] 10.222.0.20 () {60 vars in 1132 bytes} [Wed Aug 31 01:04:13 2022] POST /identity/v3/users => generated 248 bytes in 342 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:04:14.128585 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:14.184750 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a99d30-1c74-441c-840a-1e182600df0b None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:14.223011 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d6a61c-042e-4247-8642-a4903a0b6cb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:14.223690 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d6a61c-042e-4247-8642-a4903a0b6cb7 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:14.223934 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d6a61c-042e-4247-8642-a4903a0b6cb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:14.224219 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d6a61c-042e-4247-8642-a4903a0b6cb7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:14.235360 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d6a61c-042e-4247-8642-a4903a0b6cb7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:14.236891 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d6a61c-042e-4247-8642-a4903a0b6cb7 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:14.238165 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572c72b1-6119-4743-8dab-f6bafc5e3aec None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:14.257190 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application [None req-f0d6a61c-042e-4247-8642-a4903a0b6cb7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 01:04:14.257190 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:04:14.257190 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:04:14.257190 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:04:14.257190 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:04:14.257190 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:04:14.257190 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:04:14.257190 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:04:14.257190 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:04:14.257190 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:04:14.257190 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:04:14.257190 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:04:14.257190 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 01:04:14.257190 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 01:04:14.257190 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 01:04:14.257190 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 01:04:14.258726 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:04:14.258726 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:04:14.258726 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:04:14.258726 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:04:14.258726 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:04:14.258726 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:04:14.258726 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:04:14.258726 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application with Lock( Aug 31 01:04:14.258726 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:04:14.258726 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:04:14.258726 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:04:14.258726 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:04:14.258726 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:04:14.258726 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:04:14.258726 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:04:14.258726 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:04:14.258726 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 01:04:14.259568 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 01:04:14.259568 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 01:04:14.259568 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Aug 31 01:04:14.259568 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 249/475] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 01:04:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 01:04:14.265712 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d6a61c-042e-4247-8642-a4903a0b6cb7 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:14.353789 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572c72b1-6119-4743-8dab-f6bafc5e3aec None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:14.373195 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:14.389772 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d6a61c-042e-4247-8642-a4903a0b6cb7 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:14.392389 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:14.393239 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f99457-b516-4c43-9b8a-6082ff2aaaec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:14.394062 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f99457-b516-4c43-9b8a-6082ff2aaaec None admin] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:14.394526 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f99457-b516-4c43-9b8a-6082ff2aaaec None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:14.394970 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f99457-b516-4c43-9b8a-6082ff2aaaec None admin] PATH_INFO: `/v3/roles` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:14.395826 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f99457-b516-4c43-9b8a-6082ff2aaaec None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:14.397399 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f99457-b516-4c43-9b8a-6082ff2aaaec None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:14.431240 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff2caee-e9a3-499e-b207-747a618a814e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:14.431852 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff2caee-e9a3-499e-b207-747a618a814e None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:14.432129 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff2caee-e9a3-499e-b207-747a618a814e None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:14.432401 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff2caee-e9a3-499e-b207-747a618a814e None admin] PATH_INFO: `/v3/domains` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:14.433109 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff2caee-e9a3-499e-b207-747a618a814e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:04:14.433402 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff2caee-e9a3-499e-b207-747a618a814e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:14.434840 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff2caee-e9a3-499e-b207-747a618a814e None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:14.437891 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.sql.core [None req-68f99457-b516-4c43-9b8a-6082ff2aaaec None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 31 01:04:14.437891 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 31 01:04:14.437891 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [parameters: {'id': 'b109ccfb22b94c00b8fe0a4c792e18a9', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 31 01:04:14.437891 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81749) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 31 01:04:14.454966 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application [None req-68f99457-b516-4c43-9b8a-6082ff2aaaec None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 31 01:04:14.454966 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:04:14.454966 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 31 01:04:14.454966 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 31 01:04:14.454966 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 31 01:04:14.454966 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 31 01:04:14.454966 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 31 01:04:14.454966 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application result = self._query(query) Aug 31 01:04:14.454966 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 31 01:04:14.454966 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application conn.query(q) Aug 31 01:04:14.454966 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 31 01:04:14.454966 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 31 01:04:14.454966 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 31 01:04:14.454966 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application result.read() Aug 31 01:04:14.454966 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 31 01:04:14.455926 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 31 01:04:14.455926 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 31 01:04:14.455926 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 31 01:04:14.455926 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 31 01:04:14.455926 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 31 01:04:14.455926 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 31 01:04:14.455926 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 31 01:04:14.455926 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 31 01:04:14.455926 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Aug 31 01:04:14.455926 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 31 01:04:14.455926 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Aug 31 01:04:14.455926 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:04:14.455926 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 31 01:04:14.455926 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 31 01:04:14.455926 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Aug 31 01:04:14.455926 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return ref.to_dict() Aug 31 01:04:14.455926 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 31 01:04:14.455926 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application next(self.gen) Aug 31 01:04:14.456862 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 31 01:04:14.456862 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application yield resource Aug 31 01:04:14.456862 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 31 01:04:14.456862 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application next(self.gen) Aug 31 01:04:14.456862 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 31 01:04:14.456862 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application self.session.rollback() Aug 31 01:04:14.456862 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 31 01:04:14.456862 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application self.force_reraise() Aug 31 01:04:14.456862 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 31 01:04:14.456862 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application raise self.value Aug 31 01:04:14.456862 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 31 01:04:14.456862 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 31 01:04:14.456862 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 31 01:04:14.456862 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application session.commit() Aug 31 01:04:14.456862 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 31 01:04:14.456862 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 31 01:04:14.456862 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 31 01:04:14.457763 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application self._prepare_impl() Aug 31 01:04:14.457763 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 31 01:04:14.457763 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application self.session.flush() Aug 31 01:04:14.457763 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 31 01:04:14.457763 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application self._flush(objects) Aug 31 01:04:14.457763 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 31 01:04:14.457763 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 31 01:04:14.457763 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 31 01:04:14.457763 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application compat.raise_( Aug 31 01:04:14.457763 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 31 01:04:14.457763 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application raise exception Aug 31 01:04:14.457763 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 31 01:04:14.457763 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application flush_context.execute() Aug 31 01:04:14.457763 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 31 01:04:14.457763 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application rec.execute(self) Aug 31 01:04:14.457763 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 31 01:04:14.457763 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 31 01:04:14.460692 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 31 01:04:14.460692 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 31 01:04:14.460692 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 31 01:04:14.460692 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 31 01:04:14.460692 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 31 01:04:14.460692 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 31 01:04:14.460692 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 31 01:04:14.460692 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 31 01:04:14.460692 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 31 01:04:14.460692 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 31 01:04:14.460692 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 31 01:04:14.460692 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 31 01:04:14.460692 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 31 01:04:14.460692 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 31 01:04:14.460692 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 31 01:04:14.460692 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application raise exception Aug 31 01:04:14.463050 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 31 01:04:14.463050 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 31 01:04:14.463050 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 31 01:04:14.463050 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 31 01:04:14.463050 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 31 01:04:14.463050 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application result = self._query(query) Aug 31 01:04:14.463050 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 31 01:04:14.463050 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application conn.query(q) Aug 31 01:04:14.463050 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 31 01:04:14.463050 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 31 01:04:14.463050 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 31 01:04:14.463050 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application result.read() Aug 31 01:04:14.463050 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 31 01:04:14.463050 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 31 01:04:14.463050 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 31 01:04:14.463050 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 31 01:04:14.465352 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 31 01:04:14.465352 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 31 01:04:14.465352 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 31 01:04:14.465352 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 31 01:04:14.465352 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 31 01:04:14.465352 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 31 01:04:14.465352 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application [parameters: {'id': 'b109ccfb22b94c00b8fe0a4c792e18a9', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 31 01:04:14.465352 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 31 01:04:14.465352 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Aug 31 01:04:14.465352 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 01:04:14.465352 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Aug 31 01:04:14.465352 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:04:14.465352 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:04:14.465352 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:04:14.465352 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:04:14.465352 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:04:14.468308 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:04:14.468308 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:04:14.468308 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:04:14.468308 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:04:14.468308 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:04:14.468308 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:04:14.468308 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Aug 31 01:04:14.468308 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Aug 31 01:04:14.468308 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:04:14.468308 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:04:14.468308 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Aug 31 01:04:14.468308 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Aug 31 01:04:14.468308 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 31 01:04:14.468308 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 31 01:04:14.468308 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 31 01:04:14.468308 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Aug 31 01:04:14.470062 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 227/476] 10.222.0.20 () {60 vars in 1131 bytes} [Wed Aug 31 01:04:14 2022] POST /identity/v3/roles => generated 144 bytes in 219 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 31 01:04:14.470062 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f99457-b516-4c43-9b8a-6082ff2aaaec None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:14.472563 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 250/477] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 31 01:04:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:04:14.479152 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff2caee-e9a3-499e-b207-747a618a814e None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:14.596591 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f99457-b516-4c43-9b8a-6082ff2aaaec None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:14.611669 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff2caee-e9a3-499e-b207-747a618a814e None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:14.639753 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4df622-60c6-4c32-936e-6dad8d57496f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:14.640228 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4df622-60c6-4c32-936e-6dad8d57496f None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:14.640447 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4df622-60c6-4c32-936e-6dad8d57496f None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:14.640798 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4df622-60c6-4c32-936e-6dad8d57496f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:14.652277 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4df622-60c6-4c32-936e-6dad8d57496f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 01:04:14.657619 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319abc9c-8383-437f-9eca-334466f65568 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:14.657785 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application [None req-6f4df622-60c6-4c32-936e-6dad8d57496f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 01:04:14.657785 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:04:14.657785 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:04:14.657785 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:04:14.657785 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:04:14.657785 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:04:14.657785 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:04:14.657785 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:04:14.657785 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:04:14.657785 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:04:14.657785 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:04:14.657785 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:04:14.657785 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 01:04:14.657785 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 01:04:14.657785 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 01:04:14.657785 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 01:04:14.658615 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 01:04:14.658615 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 01:04:14.658615 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 01:04:14.658615 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 01:04:14.658615 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:04:14.658615 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:04:14.658615 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:04:14.658615 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:04:14.658615 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:04:14.658615 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:04:14.658615 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:04:14.658615 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application with Lock( Aug 31 01:04:14.658615 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:04:14.658615 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:04:14.658615 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:04:14.658615 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:04:14.659427 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:04:14.659427 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:04:14.659427 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:04:14.659427 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:04:14.659427 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 01:04:14.659427 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 01:04:14.659427 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 01:04:14.659427 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 01:04:14.659427 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 01:04:14.659427 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 01:04:14.659427 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 01:04:14.659427 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Aug 31 01:04:14.659427 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 228/478] 10.222.0.20 () {56 vars in 1114 bytes} [Wed Aug 31 01:04:14 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:04:14.660880 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319abc9c-8383-437f-9eca-334466f65568 None admin] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:14.661372 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319abc9c-8383-437f-9eca-334466f65568 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:14.661794 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319abc9c-8383-437f-9eca-334466f65568 None admin] PATH_INFO: `/v3/users` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:14.662596 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319abc9c-8383-437f-9eca-334466f65568 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:14.664295 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319abc9c-8383-437f-9eca-334466f65568 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:14.664654 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4df622-60c6-4c32-936e-6dad8d57496f None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:14.715996 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:14.736484 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 251/479] 10.222.0.20 () {60 vars in 1132 bytes} [Wed Aug 31 01:04:14 2022] POST /identity/v3/users => generated 249 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 01:04:14.790402 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4df622-60c6-4c32-936e-6dad8d57496f None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:14.839761 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5388975b-122f-49c6-ae39-164330c86967 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:14.840397 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5388975b-122f-49c6-ae39-164330c86967 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:14.840683 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5388975b-122f-49c6-ae39-164330c86967 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:14.840988 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5388975b-122f-49c6-ae39-164330c86967 None admin] PATH_INFO: `/v3/roles` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:14.841678 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5388975b-122f-49c6-ae39-164330c86967 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:04:14.841987 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5388975b-122f-49c6-ae39-164330c86967 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:14.843496 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5388975b-122f-49c6-ae39-164330c86967 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:14.853290 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 229/480] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Aug 31 01:04:14 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 01:04:15.749702 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accc059d-5127-4b79-b3ff-9d06ca4518f1 None None] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:15.750578 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accc059d-5127-4b79-b3ff-9d06ca4518f1 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:15.751001 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accc059d-5127-4b79-b3ff-9d06ca4518f1 None None] PATH_INFO: `/` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:15.752525 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 252/481] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:04:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:04:15.762585 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294790ef-1972-425f-9586-511b84759156 None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:15.763137 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294790ef-1972-425f-9586-511b84759156 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:15.763137 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294790ef-1972-425f-9586-511b84759156 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:15.789747 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-294790ef-1972-425f-9586-511b84759156 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:04:15.790949 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-294790ef-1972-425f-9586-511b84759156 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:04:15.879244 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 230/482] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:04:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 01:04:16.288114 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accc059d-5127-4b79-b3ff-9d06ca4518f1 None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:16.312295 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:16.382368 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2199fe-c77e-4996-8e1b-c10fd508b08b None None] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:16.382676 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2199fe-c77e-4996-8e1b-c10fd508b08b None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:16.383442 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2199fe-c77e-4996-8e1b-c10fd508b08b None None] PATH_INFO: `/` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:16.385063 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 231/483] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:04:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 01:04:16.389795 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8829e454-db83-4343-9044-05585fa23903 None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:16.390083 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8829e454-db83-4343-9044-05585fa23903 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:16.390359 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8829e454-db83-4343-9044-05585fa23903 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:16.421285 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accc059d-5127-4b79-b3ff-9d06ca4518f1 None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:16.429354 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-8829e454-db83-4343-9044-05585fa23903 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:04:16.430601 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-8829e454-db83-4343-9044-05585fa23903 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:04:16.485687 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:16.490600 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474506c4-a05c-4e40-91ae-32f4526e79f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:16.491264 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474506c4-a05c-4e40-91ae-32f4526e79f3 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:16.491554 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474506c4-a05c-4e40-91ae-32f4526e79f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:16.491827 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474506c4-a05c-4e40-91ae-32f4526e79f3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:16.521475 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474506c4-a05c-4e40-91ae-32f4526e79f3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 01:04:16.526893 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application [None req-474506c4-a05c-4e40-91ae-32f4526e79f3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 01:04:16.526893 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:04:16.526893 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:04:16.526893 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:04:16.526893 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:04:16.526893 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:04:16.526893 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:04:16.526893 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:04:16.526893 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:04:16.526893 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:04:16.526893 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:04:16.526893 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:04:16.526893 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 01:04:16.526893 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 01:04:16.526893 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 01:04:16.526893 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 01:04:16.527775 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 01:04:16.527775 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 01:04:16.527775 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 01:04:16.527775 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 01:04:16.527775 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:04:16.527775 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:04:16.527775 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:04:16.527775 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:04:16.527775 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:04:16.527775 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:04:16.527775 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:04:16.527775 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application with Lock( Aug 31 01:04:16.527775 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:04:16.527775 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:04:16.527775 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:04:16.527775 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:04:16.528563 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:04:16.528563 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:04:16.528563 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:04:16.528563 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:04:16.528563 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 01:04:16.528563 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 01:04:16.528563 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 01:04:16.528563 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 01:04:16.528563 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 01:04:16.528563 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 01:04:16.528563 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 01:04:16.528563 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Aug 31 01:04:16.528563 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 253/484] 10.222.0.20 () {56 vars in 1102 bytes} [Wed Aug 31 01:04:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:04:16.535137 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474506c4-a05c-4e40-91ae-32f4526e79f3 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:16.573468 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 232/485] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:04:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 01:04:16.661029 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:16.688209 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474506c4-a05c-4e40-91ae-32f4526e79f3 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:16.696049 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f390322c-fb89-4576-a872-46fb424552e2 None None] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:16.696329 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f390322c-fb89-4576-a872-46fb424552e2 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:16.696676 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f390322c-fb89-4576-a872-46fb424552e2 None None] PATH_INFO: `/` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:16.699713 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 233/486] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:04:16 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:04:16.704680 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bab781-c543-4597-a7ae-ffc5980e5c01 None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:16.704999 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bab781-c543-4597-a7ae-ffc5980e5c01 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:16.705281 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bab781-c543-4597-a7ae-ffc5980e5c01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:16.725010 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-a6bab781-c543-4597-a7ae-ffc5980e5c01 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:04:16.726248 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-a6bab781-c543-4597-a7ae-ffc5980e5c01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:04:16.740151 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d5138f-5f67-4ae8-bf75-7e0d29fd20d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:16.740812 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d5138f-5f67-4ae8-bf75-7e0d29fd20d7 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:16.741122 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d5138f-5f67-4ae8-bf75-7e0d29fd20d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:16.741402 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d5138f-5f67-4ae8-bf75-7e0d29fd20d7 None admin] PATH_INFO: `/v3/roles` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:16.742098 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d5138f-5f67-4ae8-bf75-7e0d29fd20d7 None admin] RBAC: Adding query filter params (name=service) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:04:16.742945 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d5138f-5f67-4ae8-bf75-7e0d29fd20d7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:16.744384 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d5138f-5f67-4ae8-bf75-7e0d29fd20d7 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:16.764710 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 254/487] 10.222.0.20 () {56 vars in 1103 bytes} [Wed Aug 31 01:04:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:04:16.777200 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d5138f-5f67-4ae8-bf75-7e0d29fd20d7 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:16.832947 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 234/488] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:04:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 01:04:16.862244 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bab781-c543-4597-a7ae-ffc5980e5c01 None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:16.986881 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d5138f-5f67-4ae8-bf75-7e0d29fd20d7 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:17.005104 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:17.016250 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bab781-c543-4597-a7ae-ffc5980e5c01 None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:17.058137 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f757bc3-4c6e-47c7-b69d-512e4742af59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:17.058760 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f757bc3-4c6e-47c7-b69d-512e4742af59 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:17.059043 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f757bc3-4c6e-47c7-b69d-512e4742af59 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:17.059321 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f757bc3-4c6e-47c7-b69d-512e4742af59 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:17.065795 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d698521d-2583-4256-b101-816fc095f4d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:17.066350 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d698521d-2583-4256-b101-816fc095f4d2 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:17.066629 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d698521d-2583-4256-b101-816fc095f4d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:17.067430 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d698521d-2583-4256-b101-816fc095f4d2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:17.080741 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f757bc3-4c6e-47c7-b69d-512e4742af59 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 01:04:17.084312 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:17.089192 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application [None req-1f757bc3-4c6e-47c7-b69d-512e4742af59 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 01:04:17.089192 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:04:17.089192 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:04:17.089192 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:04:17.089192 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:04:17.089192 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:04:17.089192 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:04:17.089192 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:04:17.089192 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:04:17.089192 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:04:17.089192 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:04:17.089192 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:04:17.089192 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 01:04:17.089192 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 01:04:17.089192 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 01:04:17.089192 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 01:04:17.090124 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 01:04:17.090124 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 01:04:17.090124 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 01:04:17.090124 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 01:04:17.090124 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:04:17.090124 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:04:17.090124 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:04:17.090124 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:04:17.090124 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:04:17.090124 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:04:17.090124 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:04:17.090124 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application with Lock( Aug 31 01:04:17.090124 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:04:17.090124 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:04:17.090124 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:04:17.090124 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:04:17.091037 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:04:17.091037 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:04:17.091037 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:04:17.091037 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:04:17.091037 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 01:04:17.091037 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 01:04:17.091037 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 01:04:17.091037 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 01:04:17.091037 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 01:04:17.091037 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 01:04:17.091037 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 01:04:17.091037 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Aug 31 01:04:17.096761 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d698521d-2583-4256-b101-816fc095f4d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:17.097703 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 235/489] 10.222.0.20 () {56 vars in 1102 bytes} [Wed Aug 31 01:04:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 01:04:17.098104 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d698521d-2583-4256-b101-816fc095f4d2 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:17.099943 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f757bc3-4c6e-47c7-b69d-512e4742af59 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:17.118995 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application [None req-d698521d-2583-4256-b101-816fc095f4d2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 01:04:17.118995 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:04:17.118995 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:04:17.118995 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:04:17.118995 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:04:17.118995 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:04:17.118995 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:04:17.118995 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:04:17.118995 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:04:17.118995 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:04:17.118995 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:04:17.118995 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:04:17.118995 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 01:04:17.118995 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 01:04:17.118995 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 01:04:17.118995 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 01:04:17.119959 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:04:17.119959 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:04:17.119959 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:04:17.119959 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:04:17.119959 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:04:17.119959 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:04:17.119959 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:04:17.119959 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application with Lock( Aug 31 01:04:17.119959 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:04:17.119959 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:04:17.119959 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:04:17.119959 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:04:17.119959 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:04:17.119959 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:04:17.119959 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:04:17.119959 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:04:17.119959 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 01:04:17.120891 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 01:04:17.120891 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 01:04:17.120891 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Aug 31 01:04:17.120891 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 255/490] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 01:04:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 344 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:04:17.125723 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d698521d-2583-4256-b101-816fc095f4d2 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:17.240074 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f757bc3-4c6e-47c7-b69d-512e4742af59 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:17.243839 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d698521d-2583-4256-b101-816fc095f4d2 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:17.275685 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198efc97-320e-4f81-abed-a3fbcda8e312 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:17.276292 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198efc97-320e-4f81-abed-a3fbcda8e312 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:17.276576 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198efc97-320e-4f81-abed-a3fbcda8e312 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:17.276887 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198efc97-320e-4f81-abed-a3fbcda8e312 None admin] PATH_INFO: `/v3/roles` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:17.277552 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198efc97-320e-4f81-abed-a3fbcda8e312 None admin] RBAC: Adding query filter params (name=service) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:04:17.277914 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198efc97-320e-4f81-abed-a3fbcda8e312 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:17.278541 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f93181-6bb9-474d-9d19-285a13745e8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:17.279171 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f93181-6bb9-474d-9d19-285a13745e8c None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:17.279263 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198efc97-320e-4f81-abed-a3fbcda8e312 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:17.279435 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f93181-6bb9-474d-9d19-285a13745e8c None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:17.279713 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f93181-6bb9-474d-9d19-285a13745e8c None admin] PATH_INFO: `/v3/domains` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:17.280352 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f93181-6bb9-474d-9d19-285a13745e8c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:04:17.280657 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f93181-6bb9-474d-9d19-285a13745e8c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:17.282092 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f93181-6bb9-474d-9d19-285a13745e8c None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:17.288004 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 236/491] 10.222.0.20 () {56 vars in 1103 bytes} [Wed Aug 31 01:04:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:04:17.291208 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f4b567-ab49-4c04-9f47-868de94b8526 None None] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:17.291705 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f4b567-ab49-4c04-9f47-868de94b8526 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:17.293038 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f4b567-ab49-4c04-9f47-868de94b8526 None None] PATH_INFO: `/` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:17.294416 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 237/492] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:04:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:04:17.299862 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 256/493] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 31 01:04:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:04:17.299862 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f93181-6bb9-474d-9d19-285a13745e8c None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:17.300080 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f4b567-ab49-4c04-9f47-868de94b8526 None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:17.316442 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:17.350660 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:17.403496 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f93181-6bb9-474d-9d19-285a13745e8c None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:17.406732 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f4b567-ab49-4c04-9f47-868de94b8526 None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:17.439447 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296421d7-bb8e-4706-8524-bd83c63c25af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:17.439637 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc910315-d132-4335-b5bf-e2b4d12ff05b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:17.440070 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296421d7-bb8e-4706-8524-bd83c63c25af None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:17.440163 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc910315-d132-4335-b5bf-e2b4d12ff05b None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:17.440348 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296421d7-bb8e-4706-8524-bd83c63c25af None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:17.440461 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc910315-d132-4335-b5bf-e2b4d12ff05b None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:17.440621 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296421d7-bb8e-4706-8524-bd83c63c25af None admin] PATH_INFO: `/v3/roles/service` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:17.440716 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc910315-d132-4335-b5bf-e2b4d12ff05b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:17.451262 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc910315-d132-4335-b5bf-e2b4d12ff05b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:17.451540 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296421d7-bb8e-4706-8524-bd83c63c25af None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 01:04:17.452740 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc910315-d132-4335-b5bf-e2b4d12ff05b None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:17.456873 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application [None req-296421d7-bb8e-4706-8524-bd83c63c25af None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 01:04:17.456873 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:04:17.456873 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:04:17.456873 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:04:17.456873 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:04:17.456873 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:04:17.456873 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:04:17.456873 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:04:17.456873 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:04:17.456873 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:04:17.456873 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:04:17.456873 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:04:17.456873 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 01:04:17.456873 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 01:04:17.456873 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 01:04:17.456873 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 01:04:17.457901 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 01:04:17.457901 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 01:04:17.457901 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 01:04:17.457901 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 01:04:17.457901 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:04:17.457901 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:04:17.457901 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:04:17.457901 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:04:17.457901 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:04:17.457901 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:04:17.457901 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:04:17.457901 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application with Lock( Aug 31 01:04:17.457901 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:04:17.457901 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:04:17.457901 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:04:17.457901 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:04:17.458742 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:04:17.458742 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:04:17.458742 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:04:17.458742 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:04:17.458742 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 01:04:17.458742 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 01:04:17.458742 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 01:04:17.458742 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 01:04:17.458742 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 01:04:17.458742 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 01:04:17.458742 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 01:04:17.458742 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Aug 31 01:04:17.458742 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 238/494] 10.222.0.20 () {56 vars in 1102 bytes} [Wed Aug 31 01:04:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:04:17.459560 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71cfd57-ea22-4d0c-a029-c0469845f082 None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:17.459787 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71cfd57-ea22-4d0c-a029-c0469845f082 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:17.460072 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71cfd57-ea22-4d0c-a029-c0469845f082 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:17.469413 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application [None req-bc910315-d132-4335-b5bf-e2b4d12ff05b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 01:04:17.469413 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:04:17.469413 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:04:17.469413 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:04:17.469413 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:04:17.469413 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:04:17.469413 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:04:17.469413 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:04:17.469413 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:04:17.469413 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:04:17.469413 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:04:17.469413 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:04:17.469413 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 01:04:17.469413 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 01:04:17.469413 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 01:04:17.469413 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 01:04:17.470469 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:04:17.470469 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:04:17.470469 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:04:17.470469 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:04:17.470469 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:04:17.470469 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:04:17.470469 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:04:17.470469 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application with Lock( Aug 31 01:04:17.470469 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:04:17.470469 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:04:17.470469 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:04:17.470469 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:04:17.470469 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:04:17.470469 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:04:17.470469 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:04:17.470469 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:04:17.470469 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 01:04:17.476393 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 01:04:17.476393 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 01:04:17.476393 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Aug 31 01:04:17.476393 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 257/495] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 01:04:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:04:17.476393 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc910315-d132-4335-b5bf-e2b4d12ff05b None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:17.477986 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-c71cfd57-ea22-4d0c-a029-c0469845f082 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:04:17.479193 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-c71cfd57-ea22-4d0c-a029-c0469845f082 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:04:17.543716 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 239/496] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:04:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 01:04:17.545095 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a278c569-ed0b-4da2-869b-391d6457a000 None None] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:17.545386 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a278c569-ed0b-4da2-869b-391d6457a000 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:17.545662 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a278c569-ed0b-4da2-869b-391d6457a000 None None] PATH_INFO: `/` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:17.547047 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 240/497] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:04:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:04:17.549475 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a278c569-ed0b-4da2-869b-391d6457a000 None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:17.558093 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:17.571835 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc910315-d132-4335-b5bf-e2b4d12ff05b None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:17.573694 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:17.609586 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6c7f8b-09af-41cf-803e-6bc1610a074e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:17.610257 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6c7f8b-09af-41cf-803e-6bc1610a074e None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:17.610616 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6c7f8b-09af-41cf-803e-6bc1610a074e None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:17.610989 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6c7f8b-09af-41cf-803e-6bc1610a074e None admin] PATH_INFO: `/v3/users/nova` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:17.619593 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6c7f8b-09af-41cf-803e-6bc1610a074e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:17.621230 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6c7f8b-09af-41cf-803e-6bc1610a074e None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:17.634686 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application [None req-af6c7f8b-09af-41cf-803e-6bc1610a074e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 31 01:04:17.634686 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:04:17.634686 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:04:17.634686 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:04:17.634686 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:04:17.634686 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:04:17.634686 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:04:17.634686 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:04:17.634686 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:04:17.634686 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:04:17.634686 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:04:17.634686 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:04:17.634686 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 01:04:17.634686 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 01:04:17.634686 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 01:04:17.634686 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 01:04:17.635621 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:04:17.635621 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:04:17.635621 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 01:04:17.635621 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 01:04:17.635621 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 01:04:17.635621 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 01:04:17.635621 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:04:17.635621 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:04:17.635621 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:04:17.635621 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:04:17.635621 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:04:17.635621 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application with Lock( Aug 31 01:04:17.635621 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:04:17.635621 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:04:17.635621 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:04:17.635621 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:04:17.635621 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:04:17.635621 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:04:17.636506 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:04:17.636506 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:04:17.636506 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 01:04:17.636506 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 01:04:17.636506 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 01:04:17.636506 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 01:04:17.636506 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 01:04:17.636506 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 01:04:17.636506 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 31 01:04:17.636506 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Aug 31 01:04:17.636506 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 258/498] 10.222.0.20 () {56 vars in 1093 bytes} [Wed Aug 31 01:04:17 2022] GET /identity/v3/users/nova => generated 82 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:04:17.637100 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6c7f8b-09af-41cf-803e-6bc1610a074e None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:17.672452 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a278c569-ed0b-4da2-869b-391d6457a000 None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:17.706069 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dad6032-c5f3-4a7a-a5d3-e5fa356f38d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:17.706663 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dad6032-c5f3-4a7a-a5d3-e5fa356f38d1 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:17.706954 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dad6032-c5f3-4a7a-a5d3-e5fa356f38d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:17.707231 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dad6032-c5f3-4a7a-a5d3-e5fa356f38d1 None admin] PATH_INFO: `/v3/roles` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:17.707894 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dad6032-c5f3-4a7a-a5d3-e5fa356f38d1 None admin] RBAC: Adding query filter params (name=service) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:04:17.708201 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dad6032-c5f3-4a7a-a5d3-e5fa356f38d1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:17.709611 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dad6032-c5f3-4a7a-a5d3-e5fa356f38d1 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:17.722227 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 241/499] 10.222.0.20 () {56 vars in 1103 bytes} [Wed Aug 31 01:04:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:04:17.722227 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e07e5c-82c5-485b-bc56-24ba64825324 None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:17.722227 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e07e5c-82c5-485b-bc56-24ba64825324 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:17.722227 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e07e5c-82c5-485b-bc56-24ba64825324 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:17.737250 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-68e07e5c-82c5-485b-bc56-24ba64825324 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:04:17.738432 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-68e07e5c-82c5-485b-bc56-24ba64825324 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:04:17.746648 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6c7f8b-09af-41cf-803e-6bc1610a074e None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:17.781639 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b0f55b-92c9-4089-8698-47b0e8324141 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:17.782273 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b0f55b-92c9-4089-8698-47b0e8324141 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:17.782518 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b0f55b-92c9-4089-8698-47b0e8324141 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:17.782788 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b0f55b-92c9-4089-8698-47b0e8324141 None admin] PATH_INFO: `/v3/domains` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:17.783429 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b0f55b-92c9-4089-8698-47b0e8324141 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:04:17.783930 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b0f55b-92c9-4089-8698-47b0e8324141 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:17.785134 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b0f55b-92c9-4089-8698-47b0e8324141 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:17.797319 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 259/500] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 31 01:04:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:04:17.798021 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b0f55b-92c9-4089-8698-47b0e8324141 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:17.806842 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:17.814152 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 242/501] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:04:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 01:04:17.815069 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e07e5c-82c5-485b-bc56-24ba64825324 None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:17.825233 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:17.910087 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b0f55b-92c9-4089-8698-47b0e8324141 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:17.921916 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e07e5c-82c5-485b-bc56-24ba64825324 None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:17.947836 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd4af19-bb6f-423d-a7fa-b139e6b7d1b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:17.948628 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd4af19-bb6f-423d-a7fa-b139e6b7d1b8 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:17.949076 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd4af19-bb6f-423d-a7fa-b139e6b7d1b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:17.949868 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd4af19-bb6f-423d-a7fa-b139e6b7d1b8 None admin] PATH_INFO: `/v3/users` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:17.950841 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd4af19-bb6f-423d-a7fa-b139e6b7d1b8 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:04:17.951310 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd4af19-bb6f-423d-a7fa-b139e6b7d1b8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:17.952909 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd4af19-bb6f-423d-a7fa-b139e6b7d1b8 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:17.955007 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b77c417-94fb-490a-b2f8-1d00b7534f56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:17.955654 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b77c417-94fb-490a-b2f8-1d00b7534f56 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:17.955931 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b77c417-94fb-490a-b2f8-1d00b7534f56 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:17.956204 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b77c417-94fb-490a-b2f8-1d00b7534f56 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:17.967425 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b77c417-94fb-490a-b2f8-1d00b7534f56 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:17.968881 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b77c417-94fb-490a-b2f8-1d00b7534f56 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:17.983005 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application [None req-9b77c417-94fb-490a-b2f8-1d00b7534f56 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 01:04:17.983005 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:04:17.983005 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:04:17.983005 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:04:17.983005 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:04:17.983005 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:04:17.983005 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:04:17.983005 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:04:17.983005 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:04:17.983005 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:04:17.983005 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:04:17.983005 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:04:17.983005 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 01:04:17.983005 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 01:04:17.983005 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 01:04:17.983005 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 01:04:17.984067 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:04:17.984067 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:04:17.984067 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:04:17.984067 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:04:17.984067 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:04:17.984067 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:04:17.984067 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:04:17.984067 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application with Lock( Aug 31 01:04:17.984067 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:04:17.984067 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:04:17.984067 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:04:17.984067 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:04:17.984067 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:04:17.984067 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:04:17.984067 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:04:17.984067 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:04:17.984067 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 01:04:17.984941 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 01:04:17.984941 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 01:04:17.984941 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Aug 31 01:04:17.984941 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 243/502] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 01:04:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:04:17.985168 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b77c417-94fb-490a-b2f8-1d00b7534f56 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:17.995225 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 260/503] 10.222.0.20 () {56 vars in 1133 bytes} [Wed Aug 31 01:04:17 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:04:17.996226 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd4af19-bb6f-423d-a7fa-b139e6b7d1b8 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:18.041310 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:18.080097 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:18.093211 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b77c417-94fb-490a-b2f8-1d00b7534f56 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:18.099709 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd4af19-bb6f-423d-a7fa-b139e6b7d1b8 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:18.126480 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4043d4-188a-4815-9a41-ba41ff184dc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:18.127107 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4043d4-188a-4815-9a41-ba41ff184dc7 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:18.127387 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4043d4-188a-4815-9a41-ba41ff184dc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:18.127669 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4043d4-188a-4815-9a41-ba41ff184dc7 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:18.135435 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4043d4-188a-4815-9a41-ba41ff184dc7 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:18.135700 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1180111f-c9f7-4af7-9c3a-5e3cd88075fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:18.136327 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1180111f-c9f7-4af7-9c3a-5e3cd88075fe None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:18.136606 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1180111f-c9f7-4af7-9c3a-5e3cd88075fe None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:18.136930 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4043d4-188a-4815-9a41-ba41ff184dc7 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:18.137028 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1180111f-c9f7-4af7-9c3a-5e3cd88075fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:18.146915 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1180111f-c9f7-4af7-9c3a-5e3cd88075fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:18.148301 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1180111f-c9f7-4af7-9c3a-5e3cd88075fe None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:18.149095 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application [None req-2d4043d4-188a-4815-9a41-ba41ff184dc7 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 31 01:04:18.149095 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:04:18.149095 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:04:18.149095 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:04:18.149095 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:04:18.149095 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:04:18.149095 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:04:18.149095 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:04:18.149095 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:04:18.149095 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:04:18.149095 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:04:18.149095 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:04:18.149095 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 01:04:18.149095 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 01:04:18.149095 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 01:04:18.149095 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 01:04:18.150658 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:04:18.150658 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:04:18.150658 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 01:04:18.150658 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 01:04:18.150658 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 01:04:18.150658 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 01:04:18.150658 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:04:18.150658 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:04:18.150658 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:04:18.150658 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:04:18.150658 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:04:18.150658 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application with Lock( Aug 31 01:04:18.150658 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:04:18.150658 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:04:18.150658 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:04:18.150658 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:04:18.150658 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:04:18.150658 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:04:18.151658 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:04:18.151658 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:04:18.151658 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 01:04:18.151658 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 01:04:18.151658 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 01:04:18.151658 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 01:04:18.151658 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 01:04:18.151658 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 01:04:18.151658 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 31 01:04:18.151658 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Aug 31 01:04:18.151658 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 244/504] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Aug 31 01:04:17 2022] GET /identity/v3/users/glance => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:04:18.151658 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4043d4-188a-4815-9a41-ba41ff184dc7 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:18.167855 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application [None req-1180111f-c9f7-4af7-9c3a-5e3cd88075fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 01:04:18.167855 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:04:18.167855 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:04:18.167855 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:04:18.167855 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:04:18.167855 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:04:18.167855 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:04:18.167855 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:04:18.167855 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:04:18.167855 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:04:18.167855 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:04:18.167855 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:04:18.167855 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 01:04:18.167855 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 01:04:18.167855 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 01:04:18.167855 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 01:04:18.168914 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:04:18.168914 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:04:18.168914 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:04:18.168914 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:04:18.168914 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:04:18.168914 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:04:18.168914 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:04:18.168914 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application with Lock( Aug 31 01:04:18.168914 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:04:18.168914 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:04:18.168914 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:04:18.168914 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:04:18.168914 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:04:18.168914 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:04:18.168914 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:04:18.168914 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:04:18.168914 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 01:04:18.169754 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 01:04:18.169754 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 01:04:18.169754 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Aug 31 01:04:18.169754 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 261/505] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 01:04:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:04:18.169754 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1180111f-c9f7-4af7-9c3a-5e3cd88075fe None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:18.253725 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4043d4-188a-4815-9a41-ba41ff184dc7 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:18.262745 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:18.267477 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1180111f-c9f7-4af7-9c3a-5e3cd88075fe None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:18.286591 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e65c8b-6aa5-46c2-88dd-b43df229132b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:18.287197 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e65c8b-6aa5-46c2-88dd-b43df229132b None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:18.287483 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e65c8b-6aa5-46c2-88dd-b43df229132b None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:18.287762 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e65c8b-6aa5-46c2-88dd-b43df229132b None admin] PATH_INFO: `/v3/domains` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:18.288401 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e65c8b-6aa5-46c2-88dd-b43df229132b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:04:18.288706 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e65c8b-6aa5-46c2-88dd-b43df229132b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:18.290157 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e65c8b-6aa5-46c2-88dd-b43df229132b None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:18.295639 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:18.300846 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f73fcb-225e-4214-b42f-08b8c5ddc0f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:18.301446 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f73fcb-225e-4214-b42f-08b8c5ddc0f1 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:18.301737 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f73fcb-225e-4214-b42f-08b8c5ddc0f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:18.301853 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 245/506] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 31 01:04:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:04:18.302028 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f73fcb-225e-4214-b42f-08b8c5ddc0f1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:18.307626 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e65c8b-6aa5-46c2-88dd-b43df229132b None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:18.370708 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f73fcb-225e-4214-b42f-08b8c5ddc0f1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:18.372191 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f73fcb-225e-4214-b42f-08b8c5ddc0f1 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:18.405117 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e65c8b-6aa5-46c2-88dd-b43df229132b None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:18.438536 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac4f6f4-d6b0-4d4e-83cb-1238fdba2b86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:18.439165 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac4f6f4-d6b0-4d4e-83cb-1238fdba2b86 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:18.439449 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac4f6f4-d6b0-4d4e-83cb-1238fdba2b86 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:18.439727 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac4f6f4-d6b0-4d4e-83cb-1238fdba2b86 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:18.449910 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac4f6f4-d6b0-4d4e-83cb-1238fdba2b86 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 01:04:18.454759 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application [None req-7ac4f6f4-d6b0-4d4e-83cb-1238fdba2b86 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 01:04:18.454759 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:04:18.454759 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:04:18.454759 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:04:18.454759 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:04:18.454759 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:04:18.454759 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:04:18.454759 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:04:18.454759 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:04:18.454759 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:04:18.454759 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:04:18.454759 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:04:18.454759 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 01:04:18.454759 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 01:04:18.454759 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 01:04:18.454759 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 01:04:18.455635 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 01:04:18.455635 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 01:04:18.455635 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 01:04:18.455635 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 01:04:18.455635 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:04:18.455635 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:04:18.455635 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:04:18.455635 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:04:18.455635 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:04:18.455635 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:04:18.455635 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:04:18.455635 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application with Lock( Aug 31 01:04:18.455635 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:04:18.455635 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:04:18.455635 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:04:18.455635 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:04:18.456488 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:04:18.456488 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:04:18.456488 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:04:18.456488 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:04:18.456488 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 01:04:18.456488 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 01:04:18.456488 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 01:04:18.456488 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 01:04:18.456488 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 01:04:18.456488 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 01:04:18.456488 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 01:04:18.456488 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Aug 31 01:04:18.456488 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 246/507] 10.222.0.20 () {56 vars in 1102 bytes} [Wed Aug 31 01:04:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:04:18.457960 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac4f6f4-d6b0-4d4e-83cb-1238fdba2b86 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:18.469305 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 262/508] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Aug 31 01:04:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 01:04:18.470138 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f73fcb-225e-4214-b42f-08b8c5ddc0f1 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:18.490055 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:18.500753 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:18.560154 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac4f6f4-d6b0-4d4e-83cb-1238fdba2b86 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:18.568036 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f73fcb-225e-4214-b42f-08b8c5ddc0f1 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:18.593900 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1613fb-ec0c-4762-b7ee-620e4355b242 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:18.594844 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1613fb-ec0c-4762-b7ee-620e4355b242 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:18.595126 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1613fb-ec0c-4762-b7ee-620e4355b242 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:18.595402 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1613fb-ec0c-4762-b7ee-620e4355b242 None admin] PATH_INFO: `/v3/users` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:18.596124 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1613fb-ec0c-4762-b7ee-620e4355b242 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:04:18.596441 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1613fb-ec0c-4762-b7ee-620e4355b242 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:18.597933 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1613fb-ec0c-4762-b7ee-620e4355b242 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:18.602116 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea718b09-35f1-4c44-be35-90e566a79c9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:18.602723 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea718b09-35f1-4c44-be35-90e566a79c9d None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:18.603004 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea718b09-35f1-4c44-be35-90e566a79c9d None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:18.603276 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea718b09-35f1-4c44-be35-90e566a79c9d None admin] PATH_INFO: `/v3/domains` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:18.603911 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea718b09-35f1-4c44-be35-90e566a79c9d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:04:18.604214 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea718b09-35f1-4c44-be35-90e566a79c9d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:18.605619 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea718b09-35f1-4c44-be35-90e566a79c9d None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:18.616469 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 263/509] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 31 01:04:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:04:18.617350 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea718b09-35f1-4c44-be35-90e566a79c9d None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:18.634868 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 247/510] 10.222.0.20 () {56 vars in 1137 bytes} [Wed Aug 31 01:04:18 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:04:18.635691 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1613fb-ec0c-4762-b7ee-620e4355b242 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:18.715626 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:18.719055 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:18.724036 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea718b09-35f1-4c44-be35-90e566a79c9d None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:18.733910 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1613fb-ec0c-4762-b7ee-620e4355b242 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:18.757332 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21aafa5a-52ea-4004-abc2-b8ed17d563f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:18.757953 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21aafa5a-52ea-4004-abc2-b8ed17d563f6 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:18.758230 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21aafa5a-52ea-4004-abc2-b8ed17d563f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:18.758501 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21aafa5a-52ea-4004-abc2-b8ed17d563f6 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:18.766028 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21aafa5a-52ea-4004-abc2-b8ed17d563f6 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:18.767303 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f06a1d4-9603-4926-84f6-faf9a858f357 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:18.767490 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21aafa5a-52ea-4004-abc2-b8ed17d563f6 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:18.767917 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f06a1d4-9603-4926-84f6-faf9a858f357 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:18.768207 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f06a1d4-9603-4926-84f6-faf9a858f357 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:18.768486 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f06a1d4-9603-4926-84f6-faf9a858f357 None admin] PATH_INFO: `/v3/roles` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:18.769190 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f06a1d4-9603-4926-84f6-faf9a858f357 None admin] RBAC: Adding query filter params (name=service) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:04:18.769499 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f06a1d4-9603-4926-84f6-faf9a858f357 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:18.770885 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f06a1d4-9603-4926-84f6-faf9a858f357 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:18.779003 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application [None req-21aafa5a-52ea-4004-abc2-b8ed17d563f6 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 31 01:04:18.779003 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:04:18.779003 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:04:18.779003 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:04:18.779003 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:04:18.779003 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:04:18.779003 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:04:18.779003 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:04:18.779003 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:04:18.779003 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:04:18.779003 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:04:18.779003 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:04:18.779003 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 01:04:18.779003 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 01:04:18.779003 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 01:04:18.779003 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 01:04:18.779917 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:04:18.779917 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:04:18.779917 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 01:04:18.779917 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 01:04:18.779917 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 01:04:18.779917 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 01:04:18.779917 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:04:18.779917 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:04:18.779917 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:04:18.779917 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:04:18.779917 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:04:18.779917 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application with Lock( Aug 31 01:04:18.779917 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:04:18.779917 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:04:18.779917 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:04:18.779917 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:04:18.779917 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:04:18.779917 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:04:18.780953 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:04:18.780953 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:04:18.780953 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 01:04:18.780953 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 01:04:18.780953 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 01:04:18.780953 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 01:04:18.780953 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 01:04:18.780953 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 01:04:18.780953 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 31 01:04:18.780953 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Aug 31 01:04:18.781500 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 248/511] 10.222.0.20 () {56 vars in 1103 bytes} [Wed Aug 31 01:04:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:04:18.781571 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 264/512] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Aug 31 01:04:18 2022] GET /identity/v3/users/cinder => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:04:18.781635 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f06a1d4-9603-4926-84f6-faf9a858f357 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:18.781699 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21aafa5a-52ea-4004-abc2-b8ed17d563f6 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:18.881414 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f06a1d4-9603-4926-84f6-faf9a858f357 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:18.882571 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21aafa5a-52ea-4004-abc2-b8ed17d563f6 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:18.918436 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a16ba99-4218-4373-b0aa-fadc00af5eba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:18.918862 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c657812-521e-4ba9-8e47-e54aba460fd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:18.919047 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a16ba99-4218-4373-b0aa-fadc00af5eba None admin] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:18.919332 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a16ba99-4218-4373-b0aa-fadc00af5eba None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:18.919487 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c657812-521e-4ba9-8e47-e54aba460fd7 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:18.919609 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a16ba99-4218-4373-b0aa-fadc00af5eba None admin] PATH_INFO: `/v3/users` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:18.919769 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c657812-521e-4ba9-8e47-e54aba460fd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:18.920046 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c657812-521e-4ba9-8e47-e54aba460fd7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:18.920257 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a16ba99-4218-4373-b0aa-fadc00af5eba None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:18.921732 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a16ba99-4218-4373-b0aa-fadc00af5eba None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:18.930683 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c657812-521e-4ba9-8e47-e54aba460fd7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:18.932137 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c657812-521e-4ba9-8e47-e54aba460fd7 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:18.940310 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:18.946798 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application [None req-4c657812-521e-4ba9-8e47-e54aba460fd7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 01:04:18.946798 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:04:18.946798 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:04:18.946798 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:04:18.946798 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:04:18.946798 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:04:18.946798 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:04:18.946798 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:04:18.946798 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:04:18.946798 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:04:18.946798 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:04:18.946798 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:04:18.946798 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 01:04:18.946798 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 01:04:18.946798 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 01:04:18.946798 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 01:04:18.947865 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:04:18.947865 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:04:18.947865 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:04:18.947865 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:04:18.947865 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:04:18.947865 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:04:18.947865 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:04:18.947865 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application with Lock( Aug 31 01:04:18.947865 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:04:18.947865 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:04:18.947865 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:04:18.947865 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:04:18.947865 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:04:18.947865 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:04:18.947865 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:04:18.947865 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:04:18.947865 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 01:04:18.949066 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 01:04:18.949066 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 01:04:18.949066 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Aug 31 01:04:18.949066 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 249/513] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 01:04:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:04:18.949066 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c657812-521e-4ba9-8e47-e54aba460fd7 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:19.024677 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 265/514] 10.222.0.20 () {60 vars in 1132 bytes} [Wed Aug 31 01:04:18 2022] POST /identity/v3/users => generated 247 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:04:19.025562 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a16ba99-4218-4373-b0aa-fadc00af5eba None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:19.037694 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:19.054972 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c657812-521e-4ba9-8e47-e54aba460fd7 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:19.095086 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e50f0d-1bc1-4148-9ee2-b31179bb137d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:19.095942 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e50f0d-1bc1-4148-9ee2-b31179bb137d None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:19.096326 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e50f0d-1bc1-4148-9ee2-b31179bb137d None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:19.096709 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e50f0d-1bc1-4148-9ee2-b31179bb137d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:19.138765 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a16ba99-4218-4373-b0aa-fadc00af5eba None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:19.172922 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:19.174972 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e50f0d-1bc1-4148-9ee2-b31179bb137d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:19.176462 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e50f0d-1bc1-4148-9ee2-b31179bb137d None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:19.177220 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290b22bf-abaf-421c-b9c6-eb0f4d12c440 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:19.177861 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290b22bf-abaf-421c-b9c6-eb0f4d12c440 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:19.178278 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290b22bf-abaf-421c-b9c6-eb0f4d12c440 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:19.178418 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290b22bf-abaf-421c-b9c6-eb0f4d12c440 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:19.190041 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290b22bf-abaf-421c-b9c6-eb0f4d12c440 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:19.191633 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290b22bf-abaf-421c-b9c6-eb0f4d12c440 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:19.208351 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application [None req-290b22bf-abaf-421c-b9c6-eb0f4d12c440 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 01:04:19.208351 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:04:19.208351 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:04:19.208351 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:04:19.208351 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:04:19.208351 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:04:19.208351 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:04:19.208351 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:04:19.208351 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:04:19.208351 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:04:19.208351 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:04:19.208351 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:04:19.208351 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 01:04:19.208351 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 01:04:19.208351 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 01:04:19.208351 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 01:04:19.209322 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:04:19.209322 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:04:19.209322 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:04:19.209322 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:04:19.209322 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:04:19.209322 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:04:19.209322 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:04:19.209322 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application with Lock( Aug 31 01:04:19.209322 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:04:19.209322 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:04:19.209322 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:04:19.209322 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:04:19.209322 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:04:19.209322 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:04:19.209322 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:04:19.209322 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:04:19.209322 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 01:04:19.210171 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 01:04:19.210171 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 01:04:19.210171 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Aug 31 01:04:19.212269 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 266/515] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 01:04:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:04:19.213241 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290b22bf-abaf-421c-b9c6-eb0f4d12c440 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:19.280629 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:19.290258 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 250/516] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Aug 31 01:04:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 01:04:19.291260 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e50f0d-1bc1-4148-9ee2-b31179bb137d None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:19.326464 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290b22bf-abaf-421c-b9c6-eb0f4d12c440 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:19.362004 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dac3a9-1784-4872-8b8d-37d072aab4aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:19.362734 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dac3a9-1784-4872-8b8d-37d072aab4aa None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:19.363093 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dac3a9-1784-4872-8b8d-37d072aab4aa None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:19.363447 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dac3a9-1784-4872-8b8d-37d072aab4aa None admin] PATH_INFO: `/v3/users` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:19.364252 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50dac3a9-1784-4872-8b8d-37d072aab4aa None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:04:19.364629 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50dac3a9-1784-4872-8b8d-37d072aab4aa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:19.366167 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50dac3a9-1784-4872-8b8d-37d072aab4aa None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:19.384492 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 267/517] 10.222.0.20 () {56 vars in 1137 bytes} [Wed Aug 31 01:04:19 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:04:19.385442 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dac3a9-1784-4872-8b8d-37d072aab4aa None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:19.399809 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:19.403709 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e50f0d-1bc1-4148-9ee2-b31179bb137d None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:19.438657 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed1cbe4-6d06-4a91-924d-b0cdb030f747 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:19.440042 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed1cbe4-6d06-4a91-924d-b0cdb030f747 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:19.440328 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed1cbe4-6d06-4a91-924d-b0cdb030f747 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:19.440605 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed1cbe4-6d06-4a91-924d-b0cdb030f747 None admin] PATH_INFO: `/v3/domains` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:19.441276 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed1cbe4-6d06-4a91-924d-b0cdb030f747 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:04:19.441584 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed1cbe4-6d06-4a91-924d-b0cdb030f747 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:19.442971 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed1cbe4-6d06-4a91-924d-b0cdb030f747 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:19.459346 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 251/518] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 31 01:04:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:04:19.459346 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed1cbe4-6d06-4a91-924d-b0cdb030f747 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:19.503502 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dac3a9-1784-4872-8b8d-37d072aab4aa None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:19.523854 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:19.542414 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f0caee-6ea6-4400-80c1-a666948d06e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:19.543069 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f0caee-6ea6-4400-80c1-a666948d06e9 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:19.543415 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f0caee-6ea6-4400-80c1-a666948d06e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:19.543902 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f0caee-6ea6-4400-80c1-a666948d06e9 None admin] PATH_INFO: `/v3/domains` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:19.544385 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f0caee-6ea6-4400-80c1-a666948d06e9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:04:19.544688 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f0caee-6ea6-4400-80c1-a666948d06e9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:19.546081 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f0caee-6ea6-4400-80c1-a666948d06e9 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:19.558509 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 268/519] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 31 01:04:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:04:19.559125 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f0caee-6ea6-4400-80c1-a666948d06e9 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:19.568513 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed1cbe4-6d06-4a91-924d-b0cdb030f747 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:19.605421 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddae72a9-e7a4-49d3-b9ed-47b6dfbafe8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:19.606056 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddae72a9-e7a4-49d3-b9ed-47b6dfbafe8f None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:19.606336 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddae72a9-e7a4-49d3-b9ed-47b6dfbafe8f None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:19.606620 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddae72a9-e7a4-49d3-b9ed-47b6dfbafe8f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:19.617351 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddae72a9-e7a4-49d3-b9ed-47b6dfbafe8f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:19.618838 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddae72a9-e7a4-49d3-b9ed-47b6dfbafe8f None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:19.632956 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application [None req-ddae72a9-e7a4-49d3-b9ed-47b6dfbafe8f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 01:04:19.632956 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:04:19.632956 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:04:19.632956 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:04:19.632956 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:04:19.632956 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:04:19.632956 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:04:19.632956 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:04:19.632956 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:04:19.632956 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:04:19.632956 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:04:19.632956 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:04:19.632956 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 01:04:19.632956 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 01:04:19.632956 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 01:04:19.632956 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 01:04:19.634020 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:04:19.634020 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:04:19.634020 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:04:19.634020 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:04:19.634020 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:04:19.634020 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:04:19.634020 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:04:19.634020 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application with Lock( Aug 31 01:04:19.634020 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:04:19.634020 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:04:19.634020 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:04:19.634020 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:04:19.634020 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:04:19.634020 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:04:19.634020 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:04:19.634020 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:04:19.634020 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 01:04:19.635039 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 01:04:19.635039 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 01:04:19.635039 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Aug 31 01:04:19.638573 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 252/520] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 01:04:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:04:19.639442 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddae72a9-e7a4-49d3-b9ed-47b6dfbafe8f None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:19.654455 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:19.668366 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f0caee-6ea6-4400-80c1-a666948d06e9 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:19.704507 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8187c635-72bf-41df-8604-89ada8b22024 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:19.705598 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8187c635-72bf-41df-8604-89ada8b22024 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:19.706085 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8187c635-72bf-41df-8604-89ada8b22024 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:19.706661 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8187c635-72bf-41df-8604-89ada8b22024 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:19.739806 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddae72a9-e7a4-49d3-b9ed-47b6dfbafe8f None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:19.757552 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:19.772376 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f062eb7-6a8d-4082-9620-d8c1fe6cfba4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:19.773021 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f062eb7-6a8d-4082-9620-d8c1fe6cfba4 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:19.773300 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f062eb7-6a8d-4082-9620-d8c1fe6cfba4 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:19.774092 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f062eb7-6a8d-4082-9620-d8c1fe6cfba4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:19.780556 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f062eb7-6a8d-4082-9620-d8c1fe6cfba4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:19.782018 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f062eb7-6a8d-4082-9620-d8c1fe6cfba4 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:19.782279 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8187c635-72bf-41df-8604-89ada8b22024 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:19.784242 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8187c635-72bf-41df-8604-89ada8b22024 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:19.794417 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application [None req-3f062eb7-6a8d-4082-9620-d8c1fe6cfba4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 01:04:19.794417 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:04:19.794417 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:04:19.794417 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:04:19.794417 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:04:19.794417 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:04:19.794417 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:04:19.794417 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:04:19.794417 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:04:19.794417 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:04:19.794417 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:04:19.794417 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:04:19.794417 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 01:04:19.794417 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 01:04:19.794417 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 01:04:19.795250 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 01:04:19.795250 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:04:19.795250 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:04:19.795250 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:04:19.795250 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:04:19.795250 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:04:19.795250 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:04:19.795250 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:04:19.795250 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application with Lock( Aug 31 01:04:19.795250 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:04:19.795250 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:04:19.795250 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:04:19.795250 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:04:19.795250 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:04:19.795250 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:04:19.795250 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:04:19.795250 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:04:19.796234 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 01:04:19.796234 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 01:04:19.796234 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 01:04:19.796234 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 01:04:19.796234 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 01:04:19.796234 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 01:04:19.796234 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 01:04:19.796234 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Aug 31 01:04:19.796234 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 253/521] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Aug 31 01:04:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:04:19.797445 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f062eb7-6a8d-4082-9620-d8c1fe6cfba4 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:19.871461 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:19.893880 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 269/522] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Aug 31 01:04:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 01:04:19.897352 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8187c635-72bf-41df-8604-89ada8b22024 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:19.908451 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f062eb7-6a8d-4082-9620-d8c1fe6cfba4 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:19.945107 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f8b280-22de-4ec8-a790-3963434835eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:19.946420 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f8b280-22de-4ec8-a790-3963434835eb None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:19.946710 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f8b280-22de-4ec8-a790-3963434835eb None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:19.947003 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f8b280-22de-4ec8-a790-3963434835eb None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:19.955403 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f8b280-22de-4ec8-a790-3963434835eb None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:19.956902 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f8b280-22de-4ec8-a790-3963434835eb None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:19.968627 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application [None req-c4f8b280-22de-4ec8-a790-3963434835eb None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 31 01:04:19.968627 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:04:19.968627 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:04:19.968627 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:04:19.968627 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:04:19.968627 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:04:19.968627 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:04:19.968627 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:04:19.968627 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:04:19.968627 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:04:19.968627 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:04:19.968627 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:04:19.968627 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 01:04:19.968627 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 01:04:19.968627 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 01:04:19.968627 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 01:04:19.969770 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:04:19.969770 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:04:19.969770 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 01:04:19.969770 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 01:04:19.969770 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 01:04:19.969770 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 01:04:19.969770 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:04:19.969770 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:04:19.969770 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:04:19.969770 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:04:19.969770 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:04:19.969770 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application with Lock( Aug 31 01:04:19.969770 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:04:19.969770 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:04:19.969770 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:04:19.969770 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:04:19.969770 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:04:19.969770 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:04:19.970784 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:04:19.970784 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:04:19.970784 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 01:04:19.970784 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 01:04:19.970784 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 01:04:19.970784 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 01:04:19.970784 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 01:04:19.970784 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 01:04:19.970784 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 31 01:04:19.970784 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Aug 31 01:04:19.970784 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 254/523] 10.222.0.20 () {56 vars in 1102 bytes} [Wed Aug 31 01:04:19 2022] GET /identity/v3/users/neutron => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:04:19.970784 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f8b280-22de-4ec8-a790-3963434835eb None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:19.983770 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:20.004316 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8187c635-72bf-41df-8604-89ada8b22024 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:20.040651 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6e7896-7faf-4dd0-8f65-1e544472f57c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:20.041398 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6e7896-7faf-4dd0-8f65-1e544472f57c None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:20.041676 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6e7896-7faf-4dd0-8f65-1e544472f57c None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:20.041989 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6e7896-7faf-4dd0-8f65-1e544472f57c None admin] PATH_INFO: `/v3/domains` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:20.042679 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6e7896-7faf-4dd0-8f65-1e544472f57c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:04:20.043030 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6e7896-7faf-4dd0-8f65-1e544472f57c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:20.044624 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6e7896-7faf-4dd0-8f65-1e544472f57c None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:20.061060 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 270/524] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 31 01:04:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:04:20.062221 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6e7896-7faf-4dd0-8f65-1e544472f57c None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:20.085843 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f8b280-22de-4ec8-a790-3963434835eb None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:20.121626 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:20.128617 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015c34c7-344b-47e2-8297-15f77d39f3d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:20.129264 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015c34c7-344b-47e2-8297-15f77d39f3d6 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:20.129552 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015c34c7-344b-47e2-8297-15f77d39f3d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:20.129833 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015c34c7-344b-47e2-8297-15f77d39f3d6 None admin] PATH_INFO: `/v3/projects` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:20.134163 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015c34c7-344b-47e2-8297-15f77d39f3d6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:04:20.134163 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015c34c7-344b-47e2-8297-15f77d39f3d6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:20.134163 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015c34c7-344b-47e2-8297-15f77d39f3d6 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:20.157913 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 255/525] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Aug 31 01:04:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:04:20.159062 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015c34c7-344b-47e2-8297-15f77d39f3d6 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:20.175732 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6e7896-7faf-4dd0-8f65-1e544472f57c None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:20.211942 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6159f8-9d00-4b87-8646-961acfe8fcdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:20.212868 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6159f8-9d00-4b87-8646-961acfe8fcdf None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:20.213145 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6159f8-9d00-4b87-8646-961acfe8fcdf None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:20.213467 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6159f8-9d00-4b87-8646-961acfe8fcdf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:20.221112 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:20.225113 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6159f8-9d00-4b87-8646-961acfe8fcdf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:20.226563 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6159f8-9d00-4b87-8646-961acfe8fcdf None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:20.242027 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application [None req-5f6159f8-9d00-4b87-8646-961acfe8fcdf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 01:04:20.242027 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:04:20.242027 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:04:20.242027 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:04:20.242027 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:04:20.242027 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:04:20.242027 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:04:20.242027 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:04:20.242027 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:04:20.242027 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:04:20.242027 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:04:20.242027 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:04:20.242027 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 01:04:20.242027 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 01:04:20.242027 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 01:04:20.242027 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 01:04:20.243590 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:04:20.243590 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:04:20.243590 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:04:20.243590 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:04:20.243590 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:04:20.243590 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:04:20.243590 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:04:20.243590 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application with Lock( Aug 31 01:04:20.243590 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:04:20.243590 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:04:20.243590 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:04:20.243590 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:04:20.243590 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:04:20.243590 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:04:20.243590 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:04:20.243590 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:04:20.243590 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 01:04:20.249370 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 01:04:20.249370 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 01:04:20.249370 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Aug 31 01:04:20.249370 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 271/526] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 01:04:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:04:20.249370 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6159f8-9d00-4b87-8646-961acfe8fcdf None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:20.265873 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015c34c7-344b-47e2-8297-15f77d39f3d6 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:20.301002 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b140071-c1a3-4869-bb1d-bc22b8ccafa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:20.301550 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b140071-c1a3-4869-bb1d-bc22b8ccafa7 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:20.301838 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b140071-c1a3-4869-bb1d-bc22b8ccafa7 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:20.302120 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b140071-c1a3-4869-bb1d-bc22b8ccafa7 None admin] PATH_INFO: `/v3/users` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:20.302851 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b140071-c1a3-4869-bb1d-bc22b8ccafa7 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:04:20.303167 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b140071-c1a3-4869-bb1d-bc22b8ccafa7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:20.305284 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b140071-c1a3-4869-bb1d-bc22b8ccafa7 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:20.323499 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 256/527] 10.222.0.20 () {56 vars in 1139 bytes} [Wed Aug 31 01:04:20 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:04:20.324562 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b140071-c1a3-4869-bb1d-bc22b8ccafa7 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:20.354698 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6159f8-9d00-4b87-8646-961acfe8fcdf None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:20.381014 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:20.389569 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b9f047-b780-4b08-8bae-12526e14a590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:20.390203 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b9f047-b780-4b08-8bae-12526e14a590 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:20.390485 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b9f047-b780-4b08-8bae-12526e14a590 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:20.390761 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b9f047-b780-4b08-8bae-12526e14a590 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:20.397598 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b9f047-b780-4b08-8bae-12526e14a590 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:20.399562 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b9f047-b780-4b08-8bae-12526e14a590 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:20.411630 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application [None req-64b9f047-b780-4b08-8bae-12526e14a590 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 01:04:20.411630 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:04:20.411630 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:04:20.411630 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:04:20.411630 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:04:20.411630 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:04:20.411630 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:04:20.411630 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:04:20.411630 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:04:20.411630 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:04:20.411630 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:04:20.411630 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:04:20.411630 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 01:04:20.411630 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 01:04:20.411630 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 01:04:20.412501 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 01:04:20.412501 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:04:20.412501 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:04:20.412501 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:04:20.412501 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:04:20.412501 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:04:20.412501 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:04:20.412501 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:04:20.412501 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application with Lock( Aug 31 01:04:20.412501 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:04:20.412501 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:04:20.412501 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:04:20.412501 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:04:20.412501 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:04:20.412501 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:04:20.412501 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:04:20.412501 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:04:20.413391 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 01:04:20.413391 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 01:04:20.413391 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 01:04:20.413391 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 01:04:20.413391 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 01:04:20.413391 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 01:04:20.413391 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 01:04:20.413391 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Aug 31 01:04:20.413391 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 272/528] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Aug 31 01:04:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:04:20.413850 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b9f047-b780-4b08-8bae-12526e14a590 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:20.444052 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b140071-c1a3-4869-bb1d-bc22b8ccafa7 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:20.471872 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:20.480701 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa09e65e-d83d-46fb-9cf9-b6f2da5b1450 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:20.481328 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa09e65e-d83d-46fb-9cf9-b6f2da5b1450 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:20.481604 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa09e65e-d83d-46fb-9cf9-b6f2da5b1450 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:20.481901 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa09e65e-d83d-46fb-9cf9-b6f2da5b1450 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:20.482630 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa09e65e-d83d-46fb-9cf9-b6f2da5b1450 None admin] RBAC: Adding query filter params (role.id=7d00a4cfa13744a69604ac2cf1a5816e, scope.project.id=60b2abc15c2849ba8a1ed92117cc981b, user.id=8744c9de803048f583d0317d01f6a540) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:04:20.482965 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa09e65e-d83d-46fb-9cf9-b6f2da5b1450 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:20.484351 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa09e65e-d83d-46fb-9cf9-b6f2da5b1450 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:20.497040 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 257/529] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Aug 31 01:04:20 2022] GET /identity/v3/role_assignments?user.id=8744c9de803048f583d0317d01f6a540&scope.project.id=60b2abc15c2849ba8a1ed92117cc981b&role.id=7d00a4cfa13744a69604ac2cf1a5816e => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:04:20.497040 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa09e65e-d83d-46fb-9cf9-b6f2da5b1450 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:20.543596 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b9f047-b780-4b08-8bae-12526e14a590 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:20.588746 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0038dcdb-dce5-46fe-8b4c-03ac3d0ebc03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:20.589395 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0038dcdb-dce5-46fe-8b4c-03ac3d0ebc03 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:20.590016 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0038dcdb-dce5-46fe-8b4c-03ac3d0ebc03 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:20.590364 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0038dcdb-dce5-46fe-8b4c-03ac3d0ebc03 None admin] PATH_INFO: `/v3/domains` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:20.592059 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0038dcdb-dce5-46fe-8b4c-03ac3d0ebc03 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:04:20.592443 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0038dcdb-dce5-46fe-8b4c-03ac3d0ebc03 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:20.594117 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0038dcdb-dce5-46fe-8b4c-03ac3d0ebc03 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:20.617044 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 273/530] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 31 01:04:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:04:20.618022 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0038dcdb-dce5-46fe-8b4c-03ac3d0ebc03 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:20.631241 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa09e65e-d83d-46fb-9cf9-b6f2da5b1450 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:20.648375 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:20.678230 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff9124e-9102-4c44-b813-40382d798919 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:20.678931 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff9124e-9102-4c44-b813-40382d798919 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:20.679224 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff9124e-9102-4c44-b813-40382d798919 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:20.679778 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff9124e-9102-4c44-b813-40382d798919 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:20.751255 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0038dcdb-dce5-46fe-8b4c-03ac3d0ebc03 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:20.756959 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff9124e-9102-4c44-b813-40382d798919 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:20.759076 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff9124e-9102-4c44-b813-40382d798919 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:20.763922 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:20.788476 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f7dfe4-8a8f-4e85-b8fa-c397e6dc5f32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:20.789159 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f7dfe4-8a8f-4e85-b8fa-c397e6dc5f32 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:20.789524 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f7dfe4-8a8f-4e85-b8fa-c397e6dc5f32 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:20.789848 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f7dfe4-8a8f-4e85-b8fa-c397e6dc5f32 None admin] PATH_INFO: `/v3/projects` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:20.790617 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f7dfe4-8a8f-4e85-b8fa-c397e6dc5f32 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:04:20.790996 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f7dfe4-8a8f-4e85-b8fa-c397e6dc5f32 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:20.792506 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f7dfe4-8a8f-4e85-b8fa-c397e6dc5f32 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:20.817257 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 274/531] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Aug 31 01:04:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:04:20.817257 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f7dfe4-8a8f-4e85-b8fa-c397e6dc5f32 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:20.888317 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 258/532] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Aug 31 01:04:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 01:04:20.889426 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff9124e-9102-4c44-b813-40382d798919 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:20.909148 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:20.953685 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f7dfe4-8a8f-4e85-b8fa-c397e6dc5f32 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:20.997420 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff9124e-9102-4c44-b813-40382d798919 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:20.997946 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262ee058-501d-446c-9ea5-4a688701e632 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:20.998332 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262ee058-501d-446c-9ea5-4a688701e632 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:20.998615 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262ee058-501d-446c-9ea5-4a688701e632 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:20.998889 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262ee058-501d-446c-9ea5-4a688701e632 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:21.006327 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262ee058-501d-446c-9ea5-4a688701e632 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:21.007862 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262ee058-501d-446c-9ea5-4a688701e632 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:21.019961 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application [None req-262ee058-501d-446c-9ea5-4a688701e632 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 01:04:21.019961 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:04:21.019961 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:04:21.019961 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:04:21.019961 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:04:21.019961 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:04:21.019961 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:04:21.019961 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:04:21.019961 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:04:21.019961 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:04:21.019961 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:04:21.019961 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:04:21.019961 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 01:04:21.019961 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 01:04:21.019961 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 01:04:21.020819 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 01:04:21.020819 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:04:21.020819 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:04:21.020819 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:04:21.020819 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:04:21.020819 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:04:21.020819 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:04:21.020819 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:04:21.020819 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application with Lock( Aug 31 01:04:21.020819 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:04:21.020819 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:04:21.020819 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:04:21.020819 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:04:21.020819 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:04:21.020819 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:04:21.020819 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:04:21.020819 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:04:21.021687 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 01:04:21.021687 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 01:04:21.021687 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 01:04:21.021687 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 01:04:21.021687 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 01:04:21.021687 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 01:04:21.021687 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 01:04:21.021687 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Aug 31 01:04:21.021687 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 275/533] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Aug 31 01:04:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:04:21.022154 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262ee058-501d-446c-9ea5-4a688701e632 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:21.045026 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a689fa41-cc0f-4aa4-8a56-c7b78818a956 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:21.045911 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a689fa41-cc0f-4aa4-8a56-c7b78818a956 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:21.046198 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a689fa41-cc0f-4aa4-8a56-c7b78818a956 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:21.046472 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a689fa41-cc0f-4aa4-8a56-c7b78818a956 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:21.047207 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a689fa41-cc0f-4aa4-8a56-c7b78818a956 None admin] RBAC: Adding query filter params (role.id=7d00a4cfa13744a69604ac2cf1a5816e, scope.project.id=60b2abc15c2849ba8a1ed92117cc981b, user.id=d8590c288028418ebe7a4163fd8039f1) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:04:21.047519 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a689fa41-cc0f-4aa4-8a56-c7b78818a956 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:21.049722 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a689fa41-cc0f-4aa4-8a56-c7b78818a956 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:21.056784 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:21.063833 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 259/534] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Aug 31 01:04:20 2022] GET /identity/v3/role_assignments?user.id=d8590c288028418ebe7a4163fd8039f1&scope.project.id=60b2abc15c2849ba8a1ed92117cc981b&role.id=7d00a4cfa13744a69604ac2cf1a5816e => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:04:21.063833 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c092b6-5aed-474e-8c28-d037a1282031 None None] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:21.063833 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c092b6-5aed-474e-8c28-d037a1282031 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:21.063833 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c092b6-5aed-474e-8c28-d037a1282031 None None] PATH_INFO: `/` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:21.063833 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 260/535] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:04:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:04:21.063833 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c092b6-5aed-474e-8c28-d037a1282031 None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:21.148441 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262ee058-501d-446c-9ea5-4a688701e632 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:21.166558 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:21.191801 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c092b6-5aed-474e-8c28-d037a1282031 None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:21.197162 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b570d9-75c9-4ecc-be0e-b788d6ecb6fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:21.197823 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b570d9-75c9-4ecc-be0e-b788d6ecb6fb None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:21.198106 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b570d9-75c9-4ecc-be0e-b788d6ecb6fb None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:21.198379 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b570d9-75c9-4ecc-be0e-b788d6ecb6fb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:21.212400 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b570d9-75c9-4ecc-be0e-b788d6ecb6fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:21.214089 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b570d9-75c9-4ecc-be0e-b788d6ecb6fb None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:21.231225 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application [None req-25b570d9-75c9-4ecc-be0e-b788d6ecb6fb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 01:04:21.231225 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:04:21.231225 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:04:21.231225 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:04:21.231225 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:04:21.231225 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:04:21.231225 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:04:21.231225 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:04:21.231225 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:04:21.231225 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:04:21.231225 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:04:21.231225 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:04:21.231225 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 01:04:21.231225 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 01:04:21.231225 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 01:04:21.231225 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 01:04:21.232411 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:04:21.232411 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:04:21.232411 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:04:21.232411 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:04:21.232411 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:04:21.232411 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:04:21.232411 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:04:21.232411 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application with Lock( Aug 31 01:04:21.232411 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:04:21.232411 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:04:21.232411 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:04:21.232411 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:04:21.232411 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:04:21.232411 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:04:21.232411 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:04:21.232411 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:04:21.232411 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 01:04:21.233631 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 01:04:21.233631 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 01:04:21.233631 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Aug 31 01:04:21.233631 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 276/536] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 01:04:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:04:21.233886 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7649d33-e267-4fd2-8d5d-71830f5b16b8 None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:21.234083 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7649d33-e267-4fd2-8d5d-71830f5b16b8 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:21.234377 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7649d33-e267-4fd2-8d5d-71830f5b16b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:21.237212 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf97b48-e10e-439f-80d8-d7ece0c8bceb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:21.238030 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf97b48-e10e-439f-80d8-d7ece0c8bceb None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:21.238499 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf97b48-e10e-439f-80d8-d7ece0c8bceb None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:21.238964 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf97b48-e10e-439f-80d8-d7ece0c8bceb None admin] PATH_INFO: `/v3/projects` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:21.239915 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf97b48-e10e-439f-80d8-d7ece0c8bceb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:04:21.240397 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf97b48-e10e-439f-80d8-d7ece0c8bceb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:21.242030 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf97b48-e10e-439f-80d8-d7ece0c8bceb None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:21.254312 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-f7649d33-e267-4fd2-8d5d-71830f5b16b8 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:04:21.255614 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-f7649d33-e267-4fd2-8d5d-71830f5b16b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:04:21.256121 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 261/537] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Aug 31 01:04:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:04:21.257382 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf97b48-e10e-439f-80d8-d7ece0c8bceb None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:21.348875 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 277/538] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:04:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 01:04:21.350524 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7649d33-e267-4fd2-8d5d-71830f5b16b8 None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:21.379958 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:21.390630 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf97b48-e10e-439f-80d8-d7ece0c8bceb None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:21.490252 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7649d33-e267-4fd2-8d5d-71830f5b16b8 None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:21.501985 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff807cd1-1a98-4114-814c-ed9c3cbe7521 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:21.502609 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff807cd1-1a98-4114-814c-ed9c3cbe7521 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:21.502892 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff807cd1-1a98-4114-814c-ed9c3cbe7521 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:21.503170 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff807cd1-1a98-4114-814c-ed9c3cbe7521 None admin] PATH_INFO: `/v3/domains` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:21.503824 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff807cd1-1a98-4114-814c-ed9c3cbe7521 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:04:21.504132 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff807cd1-1a98-4114-814c-ed9c3cbe7521 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:21.505611 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff807cd1-1a98-4114-814c-ed9c3cbe7521 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:21.520893 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:21.529018 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a232d1-8f4f-4f60-b0ac-0f0a3269e689 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:21.529643 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a232d1-8f4f-4f60-b0ac-0f0a3269e689 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:21.529925 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a232d1-8f4f-4f60-b0ac-0f0a3269e689 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:21.530198 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a232d1-8f4f-4f60-b0ac-0f0a3269e689 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:21.531011 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a232d1-8f4f-4f60-b0ac-0f0a3269e689 None admin] RBAC: Adding query filter params (role.id=7d00a4cfa13744a69604ac2cf1a5816e, scope.project.id=60b2abc15c2849ba8a1ed92117cc981b, user.id=a5c8fc69c518412ba1aad61a9f4f558c) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:04:21.537072 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a232d1-8f4f-4f60-b0ac-0f0a3269e689 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:21.537072 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a232d1-8f4f-4f60-b0ac-0f0a3269e689 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:21.537794 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 262/539] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 31 01:04:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:04:21.539658 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 278/540] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Aug 31 01:04:21 2022] GET /identity/v3/role_assignments?user.id=a5c8fc69c518412ba1aad61a9f4f558c&scope.project.id=60b2abc15c2849ba8a1ed92117cc981b&role.id=7d00a4cfa13744a69604ac2cf1a5816e => generated 259 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:04:21.545043 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff807cd1-1a98-4114-814c-ed9c3cbe7521 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:21.663551 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff807cd1-1a98-4114-814c-ed9c3cbe7521 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:21.667559 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a232d1-8f4f-4f60-b0ac-0f0a3269e689 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:21.702159 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e83a6eb-8279-43bd-bb88-7f051c4d513f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:21.702811 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e83a6eb-8279-43bd-bb88-7f051c4d513f None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:21.703096 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e83a6eb-8279-43bd-bb88-7f051c4d513f None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:21.703376 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e83a6eb-8279-43bd-bb88-7f051c4d513f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:21.711578 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e83a6eb-8279-43bd-bb88-7f051c4d513f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:21.713974 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e83a6eb-8279-43bd-bb88-7f051c4d513f None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:21.735520 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application [None req-8e83a6eb-8279-43bd-bb88-7f051c4d513f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 01:04:21.735520 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:04:21.735520 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:04:21.735520 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:04:21.735520 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:04:21.735520 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:04:21.735520 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:04:21.735520 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:04:21.735520 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:04:21.735520 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:04:21.735520 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:04:21.735520 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:04:21.735520 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 01:04:21.735520 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 01:04:21.735520 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 01:04:21.736451 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 01:04:21.736451 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:04:21.736451 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:04:21.736451 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:04:21.736451 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:04:21.736451 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:04:21.736451 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:04:21.736451 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:04:21.736451 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application with Lock( Aug 31 01:04:21.736451 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:04:21.736451 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:04:21.736451 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:04:21.736451 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:04:21.736451 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:04:21.736451 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:04:21.736451 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:04:21.736451 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:04:21.737315 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 01:04:21.737315 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 01:04:21.737315 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 01:04:21.737315 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 01:04:21.737315 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 01:04:21.737315 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 01:04:21.737315 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 01:04:21.737315 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Aug 31 01:04:21.738224 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 263/541] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Aug 31 01:04:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 01:04:21.743997 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e83a6eb-8279-43bd-bb88-7f051c4d513f None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:21.811998 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:21.842805 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:21.858954 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e83a6eb-8279-43bd-bb88-7f051c4d513f None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:21.897453 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d0323d-f352-44ef-8bef-c5b9e97c678f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:21.900960 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d0323d-f352-44ef-8bef-c5b9e97c678f None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:21.902340 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d0323d-f352-44ef-8bef-c5b9e97c678f None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:21.902821 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d0323d-f352-44ef-8bef-c5b9e97c678f None admin] PATH_INFO: `/v3/projects` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:21.904183 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d0323d-f352-44ef-8bef-c5b9e97c678f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:04:21.906876 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d0323d-f352-44ef-8bef-c5b9e97c678f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:21.908549 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d0323d-f352-44ef-8bef-c5b9e97c678f None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:21.909515 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a232d1-8f4f-4f60-b0ac-0f0a3269e689 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:21.941430 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 264/542] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Aug 31 01:04:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:04:21.957109 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d0323d-f352-44ef-8bef-c5b9e97c678f None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:21.987034 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f628bca6-32ac-40c5-b229-86e21084465d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:21.991027 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f628bca6-32ac-40c5-b229-86e21084465d None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:21.991548 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f628bca6-32ac-40c5-b229-86e21084465d None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:21.992022 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f628bca6-32ac-40c5-b229-86e21084465d None admin] PATH_INFO: `/v3/roles/service` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:22.006279 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f628bca6-32ac-40c5-b229-86e21084465d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 01:04:22.013397 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application [None req-f628bca6-32ac-40c5-b229-86e21084465d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 01:04:22.013397 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:04:22.013397 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:04:22.013397 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:04:22.013397 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:04:22.013397 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:04:22.013397 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:04:22.013397 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:04:22.013397 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:04:22.013397 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:04:22.013397 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:04:22.013397 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:04:22.013397 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 01:04:22.013397 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 01:04:22.013397 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 01:04:22.013397 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 01:04:22.014268 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 01:04:22.014268 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 01:04:22.014268 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 01:04:22.014268 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 01:04:22.014268 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:04:22.014268 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:04:22.014268 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:04:22.014268 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:04:22.014268 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:04:22.014268 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:04:22.014268 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:04:22.014268 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application with Lock( Aug 31 01:04:22.014268 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:04:22.014268 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:04:22.014268 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:04:22.014268 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:04:22.015058 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:04:22.015058 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:04:22.015058 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:04:22.015058 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:04:22.015058 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 01:04:22.015058 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 01:04:22.015058 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 01:04:22.015058 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 01:04:22.015058 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 01:04:22.015058 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 01:04:22.015058 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 01:04:22.015058 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Aug 31 01:04:22.017766 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 279/543] 10.222.0.20 () {56 vars in 1102 bytes} [Wed Aug 31 01:04:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 351 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 01:04:22.024015 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f628bca6-32ac-40c5-b229-86e21084465d None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:22.137044 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f628bca6-32ac-40c5-b229-86e21084465d None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:22.171815 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afdbc10-b5f0-42b1-a944-aa588b5f6f38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:22.172419 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afdbc10-b5f0-42b1-a944-aa588b5f6f38 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:22.172700 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afdbc10-b5f0-42b1-a944-aa588b5f6f38 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:22.173015 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afdbc10-b5f0-42b1-a944-aa588b5f6f38 None admin] PATH_INFO: `/v3/roles` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:22.173670 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afdbc10-b5f0-42b1-a944-aa588b5f6f38 None admin] RBAC: Adding query filter params (name=service) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:04:22.173972 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afdbc10-b5f0-42b1-a944-aa588b5f6f38 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:22.175330 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afdbc10-b5f0-42b1-a944-aa588b5f6f38 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:22.184444 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 280/544] 10.222.0.20 () {56 vars in 1103 bytes} [Wed Aug 31 01:04:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 01:04:22.184956 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:22.193452 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afdbc10-b5f0-42b1-a944-aa588b5f6f38 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:22.222089 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d0323d-f352-44ef-8bef-c5b9e97c678f None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:22.259824 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:22.299815 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fce27d-e559-4e47-80ec-fe0ee9c14aaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:22.300831 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fce27d-e559-4e47-80ec-fe0ee9c14aaf None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:22.301336 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fce27d-e559-4e47-80ec-fe0ee9c14aaf None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:22.301828 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fce27d-e559-4e47-80ec-fe0ee9c14aaf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:22.302861 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fce27d-e559-4e47-80ec-fe0ee9c14aaf None admin] RBAC: Adding query filter params (role.id=7d00a4cfa13744a69604ac2cf1a5816e, scope.project.id=60b2abc15c2849ba8a1ed92117cc981b, user.id=0817c3320d5d41d8bba0550c8ae8dd60) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:04:22.303519 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fce27d-e559-4e47-80ec-fe0ee9c14aaf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:22.306785 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fce27d-e559-4e47-80ec-fe0ee9c14aaf None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:22.318013 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 265/545] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Aug 31 01:04:21 2022] GET /identity/v3/role_assignments?user.id=0817c3320d5d41d8bba0550c8ae8dd60&scope.project.id=60b2abc15c2849ba8a1ed92117cc981b&role.id=7d00a4cfa13744a69604ac2cf1a5816e => generated 259 bytes in 361 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:04:22.390809 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afdbc10-b5f0-42b1-a944-aa588b5f6f38 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:22.436854 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0c766f-38b0-489a-81fa-e6de7132789d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:22.437934 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0c766f-38b0-489a-81fa-e6de7132789d None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:22.438421 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0c766f-38b0-489a-81fa-e6de7132789d None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:22.438912 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0c766f-38b0-489a-81fa-e6de7132789d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:22.451386 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0c766f-38b0-489a-81fa-e6de7132789d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:22.453304 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0c766f-38b0-489a-81fa-e6de7132789d None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:22.476550 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application [None req-eb0c766f-38b0-489a-81fa-e6de7132789d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 01:04:22.476550 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:04:22.476550 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:04:22.476550 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:04:22.476550 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:04:22.476550 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:04:22.476550 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:04:22.476550 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:04:22.476550 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:04:22.476550 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:04:22.476550 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:04:22.476550 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:04:22.476550 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 01:04:22.476550 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 01:04:22.476550 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 01:04:22.476550 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 01:04:22.478041 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:04:22.478041 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:04:22.478041 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:04:22.478041 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:04:22.478041 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:04:22.478041 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:04:22.478041 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:04:22.478041 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application with Lock( Aug 31 01:04:22.478041 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:04:22.478041 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:04:22.478041 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:04:22.478041 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:04:22.478041 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:04:22.478041 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:04:22.478041 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:04:22.478041 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:04:22.478041 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 01:04:22.478975 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 01:04:22.478975 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 01:04:22.478975 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Aug 31 01:04:22.481160 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 281/546] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 01:04:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 01:04:22.487167 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fce27d-e559-4e47-80ec-fe0ee9c14aaf None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:22.553358 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3a102b-2854-4fe5-bd2c-cc43c3ba6c52 None None] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:22.554433 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3a102b-2854-4fe5-bd2c-cc43c3ba6c52 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:22.554960 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3a102b-2854-4fe5-bd2c-cc43c3ba6c52 None None] PATH_INFO: `/` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:22.557508 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 282/547] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:04:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 01:04:22.565759 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c054e1-6306-4726-a28e-f311919f2b4b None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:22.566644 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c054e1-6306-4726-a28e-f311919f2b4b None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:22.568044 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c054e1-6306-4726-a28e-f311919f2b4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:22.609338 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-86c054e1-6306-4726-a28e-f311919f2b4b None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:04:22.611819 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-86c054e1-6306-4726-a28e-f311919f2b4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:04:22.645217 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fce27d-e559-4e47-80ec-fe0ee9c14aaf None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:22.674303 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:22.696807 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 283/548] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:04:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 01:04:22.716643 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5baa63-637c-4083-96a0-3f902b258cea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:22.717279 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5baa63-637c-4083-96a0-3f902b258cea None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:22.717561 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5baa63-637c-4083-96a0-3f902b258cea None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:22.717833 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5baa63-637c-4083-96a0-3f902b258cea None admin] PATH_INFO: `/v3/domains` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:22.718481 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5baa63-637c-4083-96a0-3f902b258cea None admin] RBAC: Adding query filter params (name=Default) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:04:22.718785 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5baa63-637c-4083-96a0-3f902b258cea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:22.720159 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5baa63-637c-4083-96a0-3f902b258cea None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:22.739983 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 266/549] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 31 01:04:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:04:22.745778 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c054e1-6306-4726-a28e-f311919f2b4b None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:22.873835 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c054e1-6306-4726-a28e-f311919f2b4b None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:22.924161 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf8be19-5aab-47fa-bde8-3cb55f0ab7de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:22.924825 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf8be19-5aab-47fa-bde8-3cb55f0ab7de None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:22.925107 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf8be19-5aab-47fa-bde8-3cb55f0ab7de None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:22.925376 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf8be19-5aab-47fa-bde8-3cb55f0ab7de None admin] PATH_INFO: `/v3/users/swift` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:22.936093 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf8be19-5aab-47fa-bde8-3cb55f0ab7de None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:22.938208 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf8be19-5aab-47fa-bde8-3cb55f0ab7de None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:22.953173 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application [None req-1cf8be19-5aab-47fa-bde8-3cb55f0ab7de None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 31 01:04:22.953173 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:04:22.953173 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:04:22.953173 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:04:22.953173 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:04:22.953173 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:04:22.953173 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:04:22.953173 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:04:22.953173 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:04:22.953173 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:04:22.953173 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:04:22.953173 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:04:22.953173 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 01:04:22.953173 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 01:04:22.953173 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 01:04:22.953173 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 01:04:22.954128 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:04:22.954128 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:04:22.954128 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 01:04:22.954128 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 01:04:22.954128 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 01:04:22.954128 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 01:04:22.954128 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:04:22.954128 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:04:22.954128 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:04:22.954128 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:04:22.954128 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:04:22.954128 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application with Lock( Aug 31 01:04:22.954128 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:04:22.954128 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:04:22.954128 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:04:22.954128 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:04:22.954128 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:04:22.954128 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:04:22.955043 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:04:22.955043 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:04:22.955043 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 01:04:22.955043 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 01:04:22.955043 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 01:04:22.955043 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 01:04:22.955043 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 01:04:22.955043 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 01:04:22.955043 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 31 01:04:22.955043 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Aug 31 01:04:22.955043 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 284/550] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Aug 31 01:04:22 2022] GET /identity/v3/users/swift => generated 83 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:04:22.962392 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5baa63-637c-4083-96a0-3f902b258cea None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:22.996664 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf8be19-5aab-47fa-bde8-3cb55f0ab7de None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:23.035725 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:23.069079 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:23.126331 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf8be19-5aab-47fa-bde8-3cb55f0ab7de None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:23.134903 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5baa63-637c-4083-96a0-3f902b258cea None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:23.163613 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6726468a-7360-47d7-b7d5-0cbff2b61db4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:23.164772 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6726468a-7360-47d7-b7d5-0cbff2b61db4 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:23.165266 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6726468a-7360-47d7-b7d5-0cbff2b61db4 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:23.165999 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6726468a-7360-47d7-b7d5-0cbff2b61db4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:23.178580 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6726468a-7360-47d7-b7d5-0cbff2b61db4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 01:04:23.184944 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application [None req-6726468a-7360-47d7-b7d5-0cbff2b61db4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 01:04:23.184944 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:04:23.184944 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:04:23.184944 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:04:23.184944 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:04:23.184944 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:04:23.184944 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:04:23.184944 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:04:23.184944 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:04:23.184944 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:04:23.184944 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:04:23.184944 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:04:23.184944 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 01:04:23.184944 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 01:04:23.184944 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 01:04:23.184944 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 01:04:23.185859 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 01:04:23.185859 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 01:04:23.185859 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 01:04:23.185859 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 01:04:23.185859 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:04:23.185859 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:04:23.185859 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:04:23.185859 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:04:23.185859 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:04:23.185859 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:04:23.185859 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:04:23.185859 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application with Lock( Aug 31 01:04:23.185859 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:04:23.185859 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:04:23.185859 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:04:23.185859 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:04:23.186670 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:04:23.186670 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:04:23.186670 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:04:23.186670 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:04:23.186670 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 01:04:23.186670 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 01:04:23.186670 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 01:04:23.186670 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 01:04:23.186670 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 01:04:23.186670 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 01:04:23.186670 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 01:04:23.186670 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Aug 31 01:04:23.189262 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 285/551] 10.222.0.20 () {56 vars in 1102 bytes} [Wed Aug 31 01:04:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 01:04:23.195783 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6726468a-7360-47d7-b7d5-0cbff2b61db4 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:23.201436 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff37f3a-ab5f-486a-95de-5d3c9c7b74a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:23.202160 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff37f3a-ab5f-486a-95de-5d3c9c7b74a4 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:23.202517 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff37f3a-ab5f-486a-95de-5d3c9c7b74a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:23.202868 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff37f3a-ab5f-486a-95de-5d3c9c7b74a4 None admin] PATH_INFO: `/v3/users` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:23.203683 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff37f3a-ab5f-486a-95de-5d3c9c7b74a4 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:04:23.204061 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff37f3a-ab5f-486a-95de-5d3c9c7b74a4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:23.205601 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff37f3a-ab5f-486a-95de-5d3c9c7b74a4 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:23.241207 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 267/552] 10.222.0.20 () {56 vars in 1135 bytes} [Wed Aug 31 01:04:22 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:04:23.249788 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff37f3a-ab5f-486a-95de-5d3c9c7b74a4 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:23.312432 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6726468a-7360-47d7-b7d5-0cbff2b61db4 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:23.316919 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:23.377022 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edad0532-c776-4f00-b8a6-9add5480ddf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:23.377936 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edad0532-c776-4f00-b8a6-9add5480ddf9 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:23.378401 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edad0532-c776-4f00-b8a6-9add5480ddf9 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:23.378868 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edad0532-c776-4f00-b8a6-9add5480ddf9 None admin] PATH_INFO: `/v3/roles` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:23.379763 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edad0532-c776-4f00-b8a6-9add5480ddf9 None admin] RBAC: Adding query filter params (name=service) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:04:23.380246 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edad0532-c776-4f00-b8a6-9add5480ddf9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:23.381986 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edad0532-c776-4f00-b8a6-9add5480ddf9 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:23.391003 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 286/553] 10.222.0.20 () {56 vars in 1103 bytes} [Wed Aug 31 01:04:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:04:23.392477 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffa7037-719f-4929-822e-5bba1791542b None None] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:23.397149 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffa7037-719f-4929-822e-5bba1791542b None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:23.399247 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffa7037-719f-4929-822e-5bba1791542b None None] PATH_INFO: `/` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:23.399542 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 287/554] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:04:23 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:04:23.400268 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffa7037-719f-4929-822e-5bba1791542b None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:23.438416 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:23.441253 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff37f3a-ab5f-486a-95de-5d3c9c7b74a4 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:23.485836 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f967029-f402-4778-afdf-94c35078c73e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:23.486546 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f967029-f402-4778-afdf-94c35078c73e None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:23.486894 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f967029-f402-4778-afdf-94c35078c73e None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:23.487258 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f967029-f402-4778-afdf-94c35078c73e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:23.516947 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffa7037-719f-4929-822e-5bba1791542b None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:23.555502 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86aeb996-8273-4f9e-946f-fbcb5d445095 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:23.556597 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86aeb996-8273-4f9e-946f-fbcb5d445095 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:23.557109 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86aeb996-8273-4f9e-946f-fbcb5d445095 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:23.557640 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86aeb996-8273-4f9e-946f-fbcb5d445095 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:23.570421 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f967029-f402-4778-afdf-94c35078c73e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:23.571433 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86aeb996-8273-4f9e-946f-fbcb5d445095 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:23.572130 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f967029-f402-4778-afdf-94c35078c73e None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:23.573362 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86aeb996-8273-4f9e-946f-fbcb5d445095 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:23.590132 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application [None req-86aeb996-8273-4f9e-946f-fbcb5d445095 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 01:04:23.590132 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:04:23.590132 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:04:23.590132 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:04:23.590132 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:04:23.590132 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:04:23.590132 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:04:23.590132 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:04:23.590132 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:04:23.590132 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:04:23.590132 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:04:23.590132 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:04:23.590132 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 01:04:23.590132 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 01:04:23.590132 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 01:04:23.590132 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 01:04:23.591068 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:04:23.591068 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:04:23.591068 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:04:23.591068 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:04:23.591068 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:04:23.591068 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:04:23.591068 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:04:23.591068 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application with Lock( Aug 31 01:04:23.591068 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:04:23.591068 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:04:23.591068 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:04:23.591068 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:04:23.591068 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:04:23.591068 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:04:23.591068 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:04:23.591068 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:04:23.591068 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 01:04:23.592031 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 01:04:23.592031 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 01:04:23.592031 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Aug 31 01:04:23.598587 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 288/555] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 01:04:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:04:23.600295 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e273cd2f-d4a5-403f-b964-c81b28427ce1 None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:23.600792 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e273cd2f-d4a5-403f-b964-c81b28427ce1 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:23.601336 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e273cd2f-d4a5-403f-b964-c81b28427ce1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:23.621408 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-e273cd2f-d4a5-403f-b964-c81b28427ce1 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:04:23.622978 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-e273cd2f-d4a5-403f-b964-c81b28427ce1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:04:23.634636 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:23.676578 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:23.680534 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 268/556] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Aug 31 01:04:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 01:04:23.681586 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f967029-f402-4778-afdf-94c35078c73e None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:23.705562 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 289/557] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:04:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 01:04:23.706633 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e273cd2f-d4a5-403f-b964-c81b28427ce1 None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:23.789616 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f967029-f402-4778-afdf-94c35078c73e None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:23.815735 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e273cd2f-d4a5-403f-b964-c81b28427ce1 None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:23.826532 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cd23fd-0e5d-4c78-8bbb-1d56ed9120aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:23.827462 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cd23fd-0e5d-4c78-8bbb-1d56ed9120aa None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:23.827915 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cd23fd-0e5d-4c78-8bbb-1d56ed9120aa None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:23.828362 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cd23fd-0e5d-4c78-8bbb-1d56ed9120aa None admin] PATH_INFO: `/v3/domains` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:23.829327 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cd23fd-0e5d-4c78-8bbb-1d56ed9120aa None admin] RBAC: Adding query filter params (name=Default) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:04:23.829803 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cd23fd-0e5d-4c78-8bbb-1d56ed9120aa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:23.831381 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cd23fd-0e5d-4c78-8bbb-1d56ed9120aa None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:23.854426 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 269/558] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 31 01:04:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:04:23.855628 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de764cc-8a7d-4351-8e16-9c36bbce83d6 None None] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:23.856133 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de764cc-8a7d-4351-8e16-9c36bbce83d6 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:23.856133 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de764cc-8a7d-4351-8e16-9c36bbce83d6 None None] PATH_INFO: `/` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:23.862212 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 270/559] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:04:23 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:04:23.863166 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de764cc-8a7d-4351-8e16-9c36bbce83d6 None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:23.868103 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dbfd15-ce75-4455-a61b-033e6e2b29a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:23.868769 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dbfd15-ce75-4455-a61b-033e6e2b29a9 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:23.869047 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dbfd15-ce75-4455-a61b-033e6e2b29a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:23.869346 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dbfd15-ce75-4455-a61b-033e6e2b29a9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:23.880886 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8dbfd15-ce75-4455-a61b-033e6e2b29a9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:23.882362 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8dbfd15-ce75-4455-a61b-033e6e2b29a9 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:23.885792 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:23.900476 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application [None req-c8dbfd15-ce75-4455-a61b-033e6e2b29a9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 01:04:23.900476 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:04:23.900476 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:04:23.900476 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:04:23.900476 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:04:23.900476 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:04:23.900476 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:04:23.900476 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:04:23.900476 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:04:23.900476 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:04:23.900476 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:04:23.900476 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:04:23.900476 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 01:04:23.900476 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 01:04:23.900476 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 01:04:23.900476 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 01:04:23.901453 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:04:23.901453 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:04:23.901453 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:04:23.901453 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:04:23.901453 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:04:23.901453 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:04:23.901453 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:04:23.901453 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application with Lock( Aug 31 01:04:23.901453 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:04:23.901453 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:04:23.901453 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:04:23.901453 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:04:23.901453 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:04:23.901453 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:04:23.901453 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:04:23.901453 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:04:23.901453 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 01:04:23.902319 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 01:04:23.902319 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 01:04:23.902319 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Aug 31 01:04:23.902319 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 290/560] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 01:04:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:04:23.907605 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc649d68-0d62-4bfd-8188-475fbfa55d4d None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:23.907927 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc649d68-0d62-4bfd-8188-475fbfa55d4d None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:23.908396 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc649d68-0d62-4bfd-8188-475fbfa55d4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:23.927710 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-dc649d68-0d62-4bfd-8188-475fbfa55d4d None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:04:23.928967 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-dc649d68-0d62-4bfd-8188-475fbfa55d4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:04:23.936462 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:23.979654 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de764cc-8a7d-4351-8e16-9c36bbce83d6 None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:24.005858 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 291/561] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:04:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 01:04:24.006926 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc649d68-0d62-4bfd-8188-475fbfa55d4d None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:24.026817 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36a8a01-d436-4041-9504-d84849038f71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:24.027459 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36a8a01-d436-4041-9504-d84849038f71 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:24.027741 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36a8a01-d436-4041-9504-d84849038f71 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:24.028016 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36a8a01-d436-4041-9504-d84849038f71 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:24.037853 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36a8a01-d436-4041-9504-d84849038f71 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:24.040141 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36a8a01-d436-4041-9504-d84849038f71 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:24.054257 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application [None req-a36a8a01-d436-4041-9504-d84849038f71 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 31 01:04:24.054257 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:04:24.054257 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:04:24.054257 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:04:24.054257 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:04:24.054257 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:04:24.054257 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:04:24.054257 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:04:24.054257 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:04:24.054257 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:04:24.054257 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:04:24.054257 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:04:24.054257 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 01:04:24.054257 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 01:04:24.054257 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 01:04:24.054257 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 01:04:24.055739 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:04:24.055739 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:04:24.055739 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 01:04:24.055739 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 01:04:24.055739 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 01:04:24.055739 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 01:04:24.055739 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:04:24.055739 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:04:24.055739 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:04:24.055739 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:04:24.055739 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:04:24.055739 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application with Lock( Aug 31 01:04:24.055739 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:04:24.055739 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:04:24.055739 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:04:24.055739 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:04:24.055739 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:04:24.055739 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:04:24.056664 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:04:24.056664 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:04:24.056664 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 01:04:24.056664 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 01:04:24.056664 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 01:04:24.056664 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 01:04:24.056664 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 01:04:24.056664 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 01:04:24.056664 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 31 01:04:24.056664 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Aug 31 01:04:24.056664 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 271/562] 10.222.0.20 () {56 vars in 1093 bytes} [Wed Aug 31 01:04:23 2022] GET /identity/v3/users/nova => generated 82 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:04:24.057293 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36a8a01-d436-4041-9504-d84849038f71 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:24.123282 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc649d68-0d62-4bfd-8188-475fbfa55d4d None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:24.154502 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:24.165329 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26f95c4-1e7f-4cb2-a3ca-ea7312b2e2e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:24.166514 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26f95c4-1e7f-4cb2-a3ca-ea7312b2e2e9 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:24.167159 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26f95c4-1e7f-4cb2-a3ca-ea7312b2e2e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:24.167703 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26f95c4-1e7f-4cb2-a3ca-ea7312b2e2e9 None admin] PATH_INFO: `/v3/domains` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:24.168650 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26f95c4-1e7f-4cb2-a3ca-ea7312b2e2e9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:04:24.169223 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26f95c4-1e7f-4cb2-a3ca-ea7312b2e2e9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:24.170974 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26f95c4-1e7f-4cb2-a3ca-ea7312b2e2e9 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:24.187211 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 292/563] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 31 01:04:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:04:24.188043 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26f95c4-1e7f-4cb2-a3ca-ea7312b2e2e9 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:24.192202 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36a8a01-d436-4041-9504-d84849038f71 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:24.223342 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:24.241435 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bb19a8-6d90-4281-add6-ed84e328e9a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:24.242619 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bb19a8-6d90-4281-add6-ed84e328e9a9 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:24.242922 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bb19a8-6d90-4281-add6-ed84e328e9a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:24.243234 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bb19a8-6d90-4281-add6-ed84e328e9a9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:24.257514 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bb19a8-6d90-4281-add6-ed84e328e9a9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 01:04:24.264042 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application [None req-04bb19a8-6d90-4281-add6-ed84e328e9a9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 01:04:24.264042 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:04:24.264042 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:04:24.264042 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:04:24.264042 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:04:24.264042 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:04:24.264042 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:04:24.264042 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:04:24.264042 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:04:24.264042 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:04:24.264042 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:04:24.264042 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:04:24.264042 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 01:04:24.264042 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 01:04:24.264042 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 01:04:24.264042 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 01:04:24.264995 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 01:04:24.264995 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 01:04:24.264995 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 01:04:24.264995 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 01:04:24.264995 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:04:24.264995 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:04:24.264995 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:04:24.264995 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:04:24.264995 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:04:24.264995 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:04:24.264995 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:04:24.264995 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application with Lock( Aug 31 01:04:24.264995 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:04:24.264995 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:04:24.264995 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:04:24.264995 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:04:24.265814 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:04:24.265814 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:04:24.265814 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:04:24.265814 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:04:24.265814 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 01:04:24.265814 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 01:04:24.265814 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 01:04:24.265814 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 01:04:24.265814 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 01:04:24.265814 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 01:04:24.265814 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 01:04:24.265814 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Aug 31 01:04:24.265814 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 272/564] 10.222.0.20 () {56 vars in 1102 bytes} [Wed Aug 31 01:04:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:04:24.267006 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bb19a8-6d90-4281-add6-ed84e328e9a9 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:24.304235 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26f95c4-1e7f-4cb2-a3ca-ea7312b2e2e9 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:24.339492 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5c11ef-b697-445f-9c18-b3e2daf174a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:24.340484 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5c11ef-b697-445f-9c18-b3e2daf174a8 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:24.341256 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5c11ef-b697-445f-9c18-b3e2daf174a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:24.341742 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5c11ef-b697-445f-9c18-b3e2daf174a8 None admin] PATH_INFO: `/v3/users` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:24.342400 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5c11ef-b697-445f-9c18-b3e2daf174a8 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:04:24.342705 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5c11ef-b697-445f-9c18-b3e2daf174a8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:24.344137 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5c11ef-b697-445f-9c18-b3e2daf174a8 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:24.363272 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 293/565] 10.222.0.20 () {56 vars in 1133 bytes} [Wed Aug 31 01:04:24 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:04:24.365583 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5c11ef-b697-445f-9c18-b3e2daf174a8 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:24.383321 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bb19a8-6d90-4281-add6-ed84e328e9a9 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:24.414427 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:24.421475 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a5e93e-485f-480c-b7b8-0f22a5b6f5e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:24.422105 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a5e93e-485f-480c-b7b8-0f22a5b6f5e8 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:24.424606 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a5e93e-485f-480c-b7b8-0f22a5b6f5e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:24.424606 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a5e93e-485f-480c-b7b8-0f22a5b6f5e8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:24.429478 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a5e93e-485f-480c-b7b8-0f22a5b6f5e8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:24.430981 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a5e93e-485f-480c-b7b8-0f22a5b6f5e8 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:24.442336 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application [None req-51a5e93e-485f-480c-b7b8-0f22a5b6f5e8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 01:04:24.442336 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:04:24.442336 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:04:24.442336 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:04:24.442336 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:04:24.442336 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:04:24.442336 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:04:24.442336 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:04:24.442336 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:04:24.442336 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:04:24.442336 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:04:24.442336 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:04:24.442336 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 01:04:24.442336 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 01:04:24.442336 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 01:04:24.443242 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 01:04:24.443242 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:04:24.443242 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:04:24.443242 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:04:24.443242 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:04:24.443242 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:04:24.443242 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:04:24.443242 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:04:24.443242 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application with Lock( Aug 31 01:04:24.443242 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:04:24.443242 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:04:24.443242 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:04:24.443242 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:04:24.443242 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:04:24.443242 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:04:24.443242 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:04:24.443242 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:04:24.444139 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 01:04:24.444139 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 01:04:24.444139 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 01:04:24.444139 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 01:04:24.444139 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 01:04:24.444139 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 01:04:24.444139 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 01:04:24.444139 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Aug 31 01:04:24.444139 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 273/566] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Aug 31 01:04:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:04:24.444610 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a5e93e-485f-480c-b7b8-0f22a5b6f5e8 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:24.474173 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5c11ef-b697-445f-9c18-b3e2daf174a8 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:24.490307 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:24.512122 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8eee45e-b011-4185-b961-acb95f2260ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:24.512745 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8eee45e-b011-4185-b961-acb95f2260ac None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:24.513083 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8eee45e-b011-4185-b961-acb95f2260ac None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:24.513306 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8eee45e-b011-4185-b961-acb95f2260ac None admin] PATH_INFO: `/v3/roles` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:24.513963 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8eee45e-b011-4185-b961-acb95f2260ac None admin] RBAC: Adding query filter params (name=service) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:04:24.514268 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8eee45e-b011-4185-b961-acb95f2260ac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:24.515637 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8eee45e-b011-4185-b961-acb95f2260ac None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:24.524463 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 294/567] 10.222.0.20 () {56 vars in 1103 bytes} [Wed Aug 31 01:04:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:04:24.525323 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8eee45e-b011-4185-b961-acb95f2260ac None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:24.555941 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a5e93e-485f-480c-b7b8-0f22a5b6f5e8 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:24.590586 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31c5606-10cb-45c1-a079-6a2b6035efe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:24.591220 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31c5606-10cb-45c1-a079-6a2b6035efe1 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:24.591502 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31c5606-10cb-45c1-a079-6a2b6035efe1 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:24.591800 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31c5606-10cb-45c1-a079-6a2b6035efe1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:24.602109 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31c5606-10cb-45c1-a079-6a2b6035efe1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 01:04:24.607374 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application [None req-f31c5606-10cb-45c1-a079-6a2b6035efe1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 01:04:24.607374 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:04:24.607374 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:04:24.607374 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:04:24.607374 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:04:24.607374 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:04:24.607374 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:04:24.607374 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:04:24.607374 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:04:24.607374 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:04:24.607374 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:04:24.607374 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:04:24.607374 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 01:04:24.607374 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 01:04:24.607374 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 01:04:24.607374 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 01:04:24.608294 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 01:04:24.608294 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 01:04:24.608294 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 01:04:24.608294 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 01:04:24.608294 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:04:24.608294 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:04:24.608294 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:04:24.608294 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:04:24.608294 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:04:24.608294 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:04:24.608294 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:04:24.608294 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application with Lock( Aug 31 01:04:24.608294 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:04:24.608294 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:04:24.608294 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:04:24.608294 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:04:24.609298 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:04:24.609298 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:04:24.609298 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:04:24.609298 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:04:24.609298 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 01:04:24.609298 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 01:04:24.609298 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 01:04:24.609298 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 01:04:24.609298 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 01:04:24.609298 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 01:04:24.609298 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 01:04:24.609298 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Aug 31 01:04:24.609298 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 274/568] 10.222.0.20 () {56 vars in 1102 bytes} [Wed Aug 31 01:04:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:04:24.611150 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31c5606-10cb-45c1-a079-6a2b6035efe1 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:24.628035 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8eee45e-b011-4185-b961-acb95f2260ac None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:24.635232 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:24.661353 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22db469-ed8c-487e-9d24-ec1bb028477c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:24.661972 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22db469-ed8c-487e-9d24-ec1bb028477c None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:24.662252 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22db469-ed8c-487e-9d24-ec1bb028477c None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:24.662522 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22db469-ed8c-487e-9d24-ec1bb028477c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:24.672621 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22db469-ed8c-487e-9d24-ec1bb028477c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:24.674137 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22db469-ed8c-487e-9d24-ec1bb028477c None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:24.688012 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application [None req-f22db469-ed8c-487e-9d24-ec1bb028477c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 01:04:24.688012 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:04:24.688012 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:04:24.688012 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:04:24.688012 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:04:24.688012 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:04:24.688012 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:04:24.688012 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:04:24.688012 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:04:24.688012 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:04:24.688012 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:04:24.688012 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:04:24.688012 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 01:04:24.688012 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 01:04:24.688012 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 01:04:24.688012 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 01:04:24.689136 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:04:24.689136 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:04:24.689136 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:04:24.689136 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:04:24.689136 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:04:24.689136 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:04:24.689136 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:04:24.689136 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application with Lock( Aug 31 01:04:24.689136 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:04:24.689136 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:04:24.689136 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:04:24.689136 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:04:24.689136 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:04:24.689136 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:04:24.689136 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:04:24.689136 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:04:24.689136 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 01:04:24.690034 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 01:04:24.690034 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 01:04:24.690034 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Aug 31 01:04:24.690034 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 295/569] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 01:04:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:04:24.690034 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22db469-ed8c-487e-9d24-ec1bb028477c None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:24.712954 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:24.715863 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31c5606-10cb-45c1-a079-6a2b6035efe1 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:24.749736 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e9f275-92ab-4c53-b65f-18958a61f4d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:24.750343 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e9f275-92ab-4c53-b65f-18958a61f4d7 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:24.750627 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e9f275-92ab-4c53-b65f-18958a61f4d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:24.750905 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e9f275-92ab-4c53-b65f-18958a61f4d7 None admin] PATH_INFO: `/v3/projects` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:24.751597 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e9f275-92ab-4c53-b65f-18958a61f4d7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:04:24.751904 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e9f275-92ab-4c53-b65f-18958a61f4d7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:24.753368 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e9f275-92ab-4c53-b65f-18958a61f4d7 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:24.765584 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 275/570] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Aug 31 01:04:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:04:24.766739 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c41ad7-1026-4971-9c26-d17198118a94 None None] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:24.767024 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c41ad7-1026-4971-9c26-d17198118a94 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:24.767297 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c41ad7-1026-4971-9c26-d17198118a94 None None] PATH_INFO: `/` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:24.768483 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 276/571] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:04:24 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:04:24.769209 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c41ad7-1026-4971-9c26-d17198118a94 None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:24.792186 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22db469-ed8c-487e-9d24-ec1bb028477c None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:24.825767 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2947db43-514e-4428-82c4-55fa211630ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:24.826395 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2947db43-514e-4428-82c4-55fa211630ca None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:24.826673 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2947db43-514e-4428-82c4-55fa211630ca None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:24.826943 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2947db43-514e-4428-82c4-55fa211630ca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:24.837174 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2947db43-514e-4428-82c4-55fa211630ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:24.838594 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2947db43-514e-4428-82c4-55fa211630ca None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:24.852194 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application [None req-2947db43-514e-4428-82c4-55fa211630ca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 01:04:24.852194 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:04:24.852194 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:04:24.852194 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:04:24.852194 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:04:24.852194 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:04:24.852194 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:04:24.852194 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:04:24.852194 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:04:24.852194 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:04:24.852194 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:04:24.852194 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:04:24.852194 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 01:04:24.852194 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 01:04:24.852194 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 01:04:24.852194 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 01:04:24.853342 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:04:24.853342 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:04:24.853342 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:04:24.853342 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:04:24.853342 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:04:24.853342 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:04:24.853342 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:04:24.853342 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application with Lock( Aug 31 01:04:24.853342 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:04:24.853342 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:04:24.853342 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:04:24.853342 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:04:24.853342 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:04:24.853342 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:04:24.853342 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:04:24.853342 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:04:24.853342 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 01:04:24.854483 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 01:04:24.854483 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 01:04:24.854483 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Aug 31 01:04:24.854483 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 296/572] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 01:04:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:04:24.854483 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2947db43-514e-4428-82c4-55fa211630ca None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:24.881500 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:24.905551 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c41ad7-1026-4971-9c26-d17198118a94 None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:24.939286 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500c8ac0-f892-4037-8ae8-bcc7ddc645ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:24.939985 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500c8ac0-f892-4037-8ae8-bcc7ddc645ef None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:24.940253 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500c8ac0-f892-4037-8ae8-bcc7ddc645ef None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:24.940530 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500c8ac0-f892-4037-8ae8-bcc7ddc645ef None admin] PATH_INFO: `/v3/roles` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:24.941225 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500c8ac0-f892-4037-8ae8-bcc7ddc645ef None admin] RBAC: Adding query filter params (name=service) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:04:24.941532 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500c8ac0-f892-4037-8ae8-bcc7ddc645ef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:24.942910 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500c8ac0-f892-4037-8ae8-bcc7ddc645ef None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:24.951550 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 277/573] 10.222.0.20 () {56 vars in 1103 bytes} [Wed Aug 31 01:04:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:04:24.952512 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500c8ac0-f892-4037-8ae8-bcc7ddc645ef None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:24.968201 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2947db43-514e-4428-82c4-55fa211630ca None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:24.976291 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:25.004676 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdf9e21-b061-43b5-85c6-aa4f6b1c4635 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:25.005336 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdf9e21-b061-43b5-85c6-aa4f6b1c4635 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:25.005619 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdf9e21-b061-43b5-85c6-aa4f6b1c4635 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:25.005899 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdf9e21-b061-43b5-85c6-aa4f6b1c4635 None admin] PATH_INFO: `/v3/domains` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:25.006572 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdf9e21-b061-43b5-85c6-aa4f6b1c4635 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:04:25.007016 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdf9e21-b061-43b5-85c6-aa4f6b1c4635 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:25.008949 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdf9e21-b061-43b5-85c6-aa4f6b1c4635 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:25.020420 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 297/574] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 31 01:04:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:04:25.021682 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317542b3-0e1b-43bc-bb18-dd5a0f5a3223 None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:25.021974 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317542b3-0e1b-43bc-bb18-dd5a0f5a3223 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:25.022249 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317542b3-0e1b-43bc-bb18-dd5a0f5a3223 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:25.040971 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-317542b3-0e1b-43bc-bb18-dd5a0f5a3223 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:04:25.042172 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-317542b3-0e1b-43bc-bb18-dd5a0f5a3223 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:04:25.056222 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500c8ac0-f892-4037-8ae8-bcc7ddc645ef None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:25.089536 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab6fd8c-3221-4838-ba7d-1d72fe4cd7fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:25.090137 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab6fd8c-3221-4838-ba7d-1d72fe4cd7fa None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:25.090426 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab6fd8c-3221-4838-ba7d-1d72fe4cd7fa None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:25.090703 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab6fd8c-3221-4838-ba7d-1d72fe4cd7fa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:25.091432 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab6fd8c-3221-4838-ba7d-1d72fe4cd7fa None admin] RBAC: Adding query filter params (role.id=7d00a4cfa13744a69604ac2cf1a5816e, scope.project.id=60b2abc15c2849ba8a1ed92117cc981b, user.id=fc1d010919ec40878ae829cc8dceec4c) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:04:25.091746 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab6fd8c-3221-4838-ba7d-1d72fe4cd7fa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:25.093224 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab6fd8c-3221-4838-ba7d-1d72fe4cd7fa None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:25.098537 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 278/575] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Aug 31 01:04:24 2022] GET /identity/v3/role_assignments?user.id=fc1d010919ec40878ae829cc8dceec4c&scope.project.id=60b2abc15c2849ba8a1ed92117cc981b&role.id=7d00a4cfa13744a69604ac2cf1a5816e => generated 259 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:04:25.099332 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab6fd8c-3221-4838-ba7d-1d72fe4cd7fa None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:25.103858 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:25.109061 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 298/576] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:04:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 01:04:25.109770 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317542b3-0e1b-43bc-bb18-dd5a0f5a3223 None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:25.208974 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:25.224326 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab6fd8c-3221-4838-ba7d-1d72fe4cd7fa None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:25.229811 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317542b3-0e1b-43bc-bb18-dd5a0f5a3223 None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:25.273491 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e523bc-ba96-453f-9bef-59d81ec3fc5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:25.274201 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e523bc-ba96-453f-9bef-59d81ec3fc5b None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:25.274310 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b57ac19-d983-461c-9356-088b50dceeed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:25.274773 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e523bc-ba96-453f-9bef-59d81ec3fc5b None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:25.274773 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e523bc-ba96-453f-9bef-59d81ec3fc5b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:25.274908 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b57ac19-d983-461c-9356-088b50dceeed None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:25.275170 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b57ac19-d983-461c-9356-088b50dceeed None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:25.275633 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b57ac19-d983-461c-9356-088b50dceeed None admin] PATH_INFO: `/v3/domains` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:25.276293 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b57ac19-d983-461c-9356-088b50dceeed None admin] RBAC: Adding query filter params (name=Default) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:04:25.277088 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b57ac19-d983-461c-9356-088b50dceeed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:25.278498 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b57ac19-d983-461c-9356-088b50dceeed None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:25.285942 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e523bc-ba96-453f-9bef-59d81ec3fc5b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:25.287566 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e523bc-ba96-453f-9bef-59d81ec3fc5b None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:25.297896 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 279/577] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 31 01:04:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:04:25.298660 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b57ac19-d983-461c-9356-088b50dceeed None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:25.304075 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application [None req-89e523bc-ba96-453f-9bef-59d81ec3fc5b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 01:04:25.304075 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:04:25.304075 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:04:25.304075 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:04:25.304075 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:04:25.304075 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:04:25.304075 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:04:25.304075 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:04:25.304075 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:04:25.304075 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:04:25.304075 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:04:25.304075 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:04:25.304075 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 01:04:25.304075 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 01:04:25.304075 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 01:04:25.304075 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 01:04:25.305714 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:04:25.305714 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:04:25.305714 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:04:25.305714 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:04:25.305714 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:04:25.305714 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:04:25.305714 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:04:25.305714 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application with Lock( Aug 31 01:04:25.305714 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:04:25.305714 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:04:25.305714 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:04:25.305714 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:04:25.305714 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:04:25.305714 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:04:25.305714 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:04:25.305714 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:04:25.305714 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 01:04:25.306598 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 01:04:25.306598 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 01:04:25.306598 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Aug 31 01:04:25.306598 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 299/578] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 01:04:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:04:25.306838 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e523bc-ba96-453f-9bef-59d81ec3fc5b None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:25.369232 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:25.426317 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b57ac19-d983-461c-9356-088b50dceeed None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:25.434189 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e523bc-ba96-453f-9bef-59d81ec3fc5b None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:25.463877 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6b9062-2dff-4869-921d-c455fe0eeaba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:25.464438 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6b9062-2dff-4869-921d-c455fe0eeaba None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:25.464735 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6b9062-2dff-4869-921d-c455fe0eeaba None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:25.465025 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6b9062-2dff-4869-921d-c455fe0eeaba None admin] PATH_INFO: `/v3/projects/service` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:25.470020 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb244d6-8364-43f9-b049-7299a10a9031 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:25.470718 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb244d6-8364-43f9-b049-7299a10a9031 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:25.471157 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb244d6-8364-43f9-b049-7299a10a9031 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:25.471439 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb244d6-8364-43f9-b049-7299a10a9031 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:25.472044 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6b9062-2dff-4869-921d-c455fe0eeaba None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:25.473527 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6b9062-2dff-4869-921d-c455fe0eeaba None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:25.479837 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb244d6-8364-43f9-b049-7299a10a9031 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:25.481305 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb244d6-8364-43f9-b049-7299a10a9031 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:25.485712 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application [None req-1d6b9062-2dff-4869-921d-c455fe0eeaba None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 01:04:25.485712 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:04:25.485712 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:04:25.485712 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:04:25.485712 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:04:25.485712 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:04:25.485712 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:04:25.485712 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:04:25.485712 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:04:25.485712 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:04:25.485712 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:04:25.485712 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:04:25.485712 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 01:04:25.485712 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 01:04:25.485712 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 01:04:25.486776 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 01:04:25.486776 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:04:25.486776 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:04:25.486776 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:04:25.486776 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:04:25.486776 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:04:25.486776 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:04:25.486776 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:04:25.486776 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application with Lock( Aug 31 01:04:25.486776 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:04:25.486776 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:04:25.486776 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:04:25.486776 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:04:25.486776 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:04:25.486776 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:04:25.486776 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:04:25.486776 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:04:25.487901 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 01:04:25.487901 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 01:04:25.487901 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 01:04:25.487901 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 01:04:25.487901 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 01:04:25.487901 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 01:04:25.487901 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 01:04:25.487901 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Aug 31 01:04:25.492631 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 280/579] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Aug 31 01:04:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:04:25.493267 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6b9062-2dff-4869-921d-c455fe0eeaba None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:25.494221 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application [None req-5eb244d6-8364-43f9-b049-7299a10a9031 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 31 01:04:25.494221 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:04:25.494221 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:04:25.494221 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:04:25.494221 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:04:25.494221 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:04:25.494221 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:04:25.494221 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:04:25.494221 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:04:25.494221 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:04:25.494221 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:04:25.494221 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:04:25.494221 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 01:04:25.494221 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 01:04:25.494221 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 01:04:25.494221 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 01:04:25.495295 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:04:25.495295 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:04:25.495295 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 01:04:25.495295 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 01:04:25.495295 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 01:04:25.495295 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 01:04:25.495295 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:04:25.495295 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:04:25.495295 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:04:25.495295 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:04:25.495295 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:04:25.495295 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application with Lock( Aug 31 01:04:25.495295 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:04:25.495295 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:04:25.495295 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:04:25.495295 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:04:25.495295 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:04:25.495295 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:04:25.496479 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:04:25.496479 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:04:25.496479 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 01:04:25.496479 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 01:04:25.496479 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 01:04:25.496479 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 01:04:25.496479 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 01:04:25.496479 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 01:04:25.496479 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 31 01:04:25.496479 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Aug 31 01:04:25.497434 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 300/580] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Aug 31 01:04:25 2022] GET /identity/v3/users/glance => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:04:25.501945 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb244d6-8364-43f9-b049-7299a10a9031 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:25.504131 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:25.601173 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6b9062-2dff-4869-921d-c455fe0eeaba None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:25.605825 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb244d6-8364-43f9-b049-7299a10a9031 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:25.619175 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:25.641987 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00875a07-e47e-4ae2-ae00-e9608d12a3de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:25.642746 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00875a07-e47e-4ae2-ae00-e9608d12a3de None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:25.643141 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00875a07-e47e-4ae2-ae00-e9608d12a3de None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:25.643522 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00875a07-e47e-4ae2-ae00-e9608d12a3de None admin] PATH_INFO: `/v3/domains` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:25.644278 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00875a07-e47e-4ae2-ae00-e9608d12a3de None admin] RBAC: Adding query filter params (name=Default) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:04:25.644679 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00875a07-e47e-4ae2-ae00-e9608d12a3de None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:25.646262 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00875a07-e47e-4ae2-ae00-e9608d12a3de None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:25.647169 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b991257-4e0f-435e-8902-b85ce37ec657 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:25.648044 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b991257-4e0f-435e-8902-b85ce37ec657 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:25.648961 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b991257-4e0f-435e-8902-b85ce37ec657 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:25.649335 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b991257-4e0f-435e-8902-b85ce37ec657 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:25.666177 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 281/581] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 31 01:04:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:04:25.666177 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00875a07-e47e-4ae2-ae00-e9608d12a3de None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:25.667890 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b991257-4e0f-435e-8902-b85ce37ec657 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 01:04:25.674454 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application [None req-0b991257-4e0f-435e-8902-b85ce37ec657 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 01:04:25.674454 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:04:25.674454 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:04:25.674454 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:04:25.674454 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:04:25.674454 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:04:25.674454 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:04:25.674454 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:04:25.674454 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:04:25.674454 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:04:25.674454 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:04:25.674454 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:04:25.674454 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 01:04:25.674454 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 01:04:25.674454 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 01:04:25.674454 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 01:04:25.675550 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 01:04:25.675550 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 01:04:25.675550 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 01:04:25.675550 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 01:04:25.675550 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:04:25.675550 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:04:25.675550 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:04:25.675550 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:04:25.675550 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:04:25.675550 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:04:25.675550 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:04:25.675550 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application with Lock( Aug 31 01:04:25.675550 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:04:25.675550 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:04:25.675550 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:04:25.675550 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:04:25.681199 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:04:25.681199 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:04:25.681199 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:04:25.681199 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:04:25.681199 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 01:04:25.681199 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 01:04:25.681199 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 01:04:25.681199 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 01:04:25.681199 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 01:04:25.681199 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 01:04:25.681199 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 01:04:25.681199 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Aug 31 01:04:25.681199 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 301/582] 10.222.0.20 () {56 vars in 1102 bytes} [Wed Aug 31 01:04:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:04:25.681199 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b991257-4e0f-435e-8902-b85ce37ec657 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:25.744471 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:25.779652 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00875a07-e47e-4ae2-ae00-e9608d12a3de None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:25.785479 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b991257-4e0f-435e-8902-b85ce37ec657 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:25.817146 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a085980-10c0-4532-977d-6d2d28dc3d7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:25.817751 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a085980-10c0-4532-977d-6d2d28dc3d7d None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:25.819529 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a085980-10c0-4532-977d-6d2d28dc3d7d None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:25.819529 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a085980-10c0-4532-977d-6d2d28dc3d7d None admin] PATH_INFO: `/v3/projects` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:25.819529 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a085980-10c0-4532-977d-6d2d28dc3d7d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:04:25.819529 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a085980-10c0-4532-977d-6d2d28dc3d7d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:25.820752 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a085980-10c0-4532-977d-6d2d28dc3d7d None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:25.821290 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5a3cb2-9305-4b22-94f8-0e9d02eedc87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:25.821958 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5a3cb2-9305-4b22-94f8-0e9d02eedc87 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:25.822309 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5a3cb2-9305-4b22-94f8-0e9d02eedc87 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:25.822652 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5a3cb2-9305-4b22-94f8-0e9d02eedc87 None admin] PATH_INFO: `/v3/users` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:25.823448 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5a3cb2-9305-4b22-94f8-0e9d02eedc87 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:04:25.823827 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5a3cb2-9305-4b22-94f8-0e9d02eedc87 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:25.825596 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5a3cb2-9305-4b22-94f8-0e9d02eedc87 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:25.832714 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 282/583] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Aug 31 01:04:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:04:25.833531 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a085980-10c0-4532-977d-6d2d28dc3d7d None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:25.849657 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 302/584] 10.222.0.20 () {56 vars in 1137 bytes} [Wed Aug 31 01:04:25 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:04:25.855229 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5a3cb2-9305-4b22-94f8-0e9d02eedc87 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:25.929894 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:25.970389 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a085980-10c0-4532-977d-6d2d28dc3d7d None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:25.995822 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:26.001596 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5a3cb2-9305-4b22-94f8-0e9d02eedc87 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:26.004493 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b49cefa-2e6d-43a9-9796-6080ebbfd55b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:26.008478 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b49cefa-2e6d-43a9-9796-6080ebbfd55b None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:26.008478 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b49cefa-2e6d-43a9-9796-6080ebbfd55b None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:26.008478 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b49cefa-2e6d-43a9-9796-6080ebbfd55b None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:26.014108 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b49cefa-2e6d-43a9-9796-6080ebbfd55b None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:26.015582 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b49cefa-2e6d-43a9-9796-6080ebbfd55b None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:26.028457 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application [None req-2b49cefa-2e6d-43a9-9796-6080ebbfd55b None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 31 01:04:26.028457 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:04:26.028457 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:04:26.028457 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:04:26.028457 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:04:26.028457 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:04:26.028457 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:04:26.028457 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:04:26.028457 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:04:26.028457 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:04:26.028457 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:04:26.028457 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:04:26.028457 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 01:04:26.028457 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 01:04:26.028457 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 01:04:26.028457 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 01:04:26.029451 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:04:26.029451 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:04:26.029451 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 01:04:26.029451 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 01:04:26.029451 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 01:04:26.029451 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 01:04:26.029451 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:04:26.029451 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:04:26.029451 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:04:26.029451 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:04:26.029451 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:04:26.029451 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application with Lock( Aug 31 01:04:26.029451 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:04:26.029451 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:04:26.029451 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:04:26.029451 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:04:26.029451 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:04:26.029451 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:04:26.030409 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:04:26.030409 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:04:26.030409 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 01:04:26.030409 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 01:04:26.030409 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 01:04:26.030409 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 01:04:26.030409 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 01:04:26.030409 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 01:04:26.030409 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 31 01:04:26.030409 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Aug 31 01:04:26.030409 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 283/585] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Aug 31 01:04:25 2022] GET /identity/v3/users/cinder => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:04:26.030409 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b49cefa-2e6d-43a9-9796-6080ebbfd55b None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:26.041953 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be359e10-d328-4883-993c-cde398a9d2b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:26.042567 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be359e10-d328-4883-993c-cde398a9d2b8 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:26.042849 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be359e10-d328-4883-993c-cde398a9d2b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:26.043126 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be359e10-d328-4883-993c-cde398a9d2b8 None admin] PATH_INFO: `/v3/roles` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:26.043785 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be359e10-d328-4883-993c-cde398a9d2b8 None admin] RBAC: Adding query filter params (name=service) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:04:26.044089 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be359e10-d328-4883-993c-cde398a9d2b8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:26.045500 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be359e10-d328-4883-993c-cde398a9d2b8 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:26.054117 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 303/586] 10.222.0.20 () {56 vars in 1103 bytes} [Wed Aug 31 01:04:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:04:26.054117 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be359e10-d328-4883-993c-cde398a9d2b8 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:26.142752 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b49cefa-2e6d-43a9-9796-6080ebbfd55b None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:26.160187 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:26.169809 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be359e10-d328-4883-993c-cde398a9d2b8 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:26.179611 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8156eb4b-87cc-41d9-9861-b7de9c282bcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:26.180317 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8156eb4b-87cc-41d9-9861-b7de9c282bcb None admin] REQUEST_METHOD: `PUT` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:26.180541 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8156eb4b-87cc-41d9-9861-b7de9c282bcb None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:26.180841 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8156eb4b-87cc-41d9-9861-b7de9c282bcb None admin] PATH_INFO: `/v3/projects/60b2abc15c2849ba8a1ed92117cc981b/users/8744c9de803048f583d0317d01f6a540/roles/7d00a4cfa13744a69604ac2cf1a5816e` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:26.196499 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8156eb4b-87cc-41d9-9861-b7de9c282bcb None admin] RBAC: Authorizing `identity:create_grant(project_id=60b2abc15c2849ba8a1ed92117cc981b, user_id=8744c9de803048f583d0317d01f6a540, role_id=7d00a4cfa13744a69604ac2cf1a5816e)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:26.198306 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8156eb4b-87cc-41d9-9861-b7de9c282bcb None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:26.204403 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021bb794-2edc-49af-ab97-6358cbc96c1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:26.205059 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021bb794-2edc-49af-ab97-6358cbc96c1f None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:26.205339 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021bb794-2edc-49af-ab97-6358cbc96c1f None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:26.205609 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021bb794-2edc-49af-ab97-6358cbc96c1f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:26.218453 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021bb794-2edc-49af-ab97-6358cbc96c1f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:26.218814 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021bb794-2edc-49af-ab97-6358cbc96c1f None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:26.233828 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application [None req-021bb794-2edc-49af-ab97-6358cbc96c1f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 01:04:26.233828 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:04:26.233828 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:04:26.233828 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:04:26.233828 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:04:26.233828 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:04:26.233828 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:04:26.233828 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:04:26.233828 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:04:26.233828 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:04:26.233828 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:04:26.233828 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:04:26.233828 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 01:04:26.233828 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 01:04:26.233828 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 01:04:26.233828 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 01:04:26.234965 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:04:26.234965 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:04:26.234965 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:04:26.234965 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:04:26.234965 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:04:26.234965 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:04:26.234965 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:04:26.234965 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application with Lock( Aug 31 01:04:26.234965 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:04:26.234965 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:04:26.234965 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:04:26.234965 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:04:26.234965 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:04:26.234965 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:04:26.234965 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:04:26.234965 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:04:26.234965 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 01:04:26.235846 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 01:04:26.235846 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 01:04:26.235846 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Aug 31 01:04:26.235846 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 304/587] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 01:04:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:04:26.235846 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021bb794-2edc-49af-ab97-6358cbc96c1f None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:26.281226 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 284/588] 10.222.0.20 () {58 vars in 1439 bytes} [Wed Aug 31 01:04:26 2022] PUT /identity/v3/projects/60b2abc15c2849ba8a1ed92117cc981b/users/8744c9de803048f583d0317d01f6a540/roles/7d00a4cfa13744a69604ac2cf1a5816e => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:04:26.282681 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8156eb4b-87cc-41d9-9861-b7de9c282bcb None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:26.327406 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:26.363185 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021bb794-2edc-49af-ab97-6358cbc96c1f None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:26.408163 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8156eb4b-87cc-41d9-9861-b7de9c282bcb None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:26.413490 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d82020f-25fb-4343-b600-f405826f184d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:26.414156 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d82020f-25fb-4343-b600-f405826f184d None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:26.414437 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d82020f-25fb-4343-b600-f405826f184d None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:26.414711 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d82020f-25fb-4343-b600-f405826f184d None admin] PATH_INFO: `/v3/users` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:26.415449 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d82020f-25fb-4343-b600-f405826f184d None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:04:26.415755 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d82020f-25fb-4343-b600-f405826f184d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:26.417247 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d82020f-25fb-4343-b600-f405826f184d None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:26.443406 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 305/589] 10.222.0.20 () {56 vars in 1137 bytes} [Wed Aug 31 01:04:26 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:04:26.444445 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d82020f-25fb-4343-b600-f405826f184d None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:26.451757 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1982ba9a-763c-4853-8301-7e07a6d058c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:26.452120 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:26.452596 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1982ba9a-763c-4853-8301-7e07a6d058c6 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:26.452959 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1982ba9a-763c-4853-8301-7e07a6d058c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:26.453235 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1982ba9a-763c-4853-8301-7e07a6d058c6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:26.465131 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1982ba9a-763c-4853-8301-7e07a6d058c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:26.466876 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1982ba9a-763c-4853-8301-7e07a6d058c6 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:26.486729 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application [None req-1982ba9a-763c-4853-8301-7e07a6d058c6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 01:04:26.486729 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:04:26.486729 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:04:26.486729 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:04:26.486729 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:04:26.486729 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:04:26.486729 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:04:26.486729 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:04:26.486729 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:04:26.486729 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:04:26.486729 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:04:26.486729 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:04:26.486729 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 01:04:26.486729 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 01:04:26.486729 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 01:04:26.486729 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 01:04:26.487791 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:04:26.487791 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:04:26.487791 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:04:26.487791 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:04:26.487791 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:04:26.487791 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:04:26.487791 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:04:26.487791 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application with Lock( Aug 31 01:04:26.487791 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:04:26.487791 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:04:26.487791 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:04:26.487791 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:04:26.487791 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:04:26.487791 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:04:26.487791 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:04:26.487791 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:04:26.487791 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 01:04:26.488675 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 01:04:26.488675 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 01:04:26.488675 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Aug 31 01:04:26.488675 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 285/590] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 01:04:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:04:26.488675 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1982ba9a-763c-4853-8301-7e07a6d058c6 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:26.584887 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d82020f-25fb-4343-b600-f405826f184d None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:26.620695 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144e6a19-b618-4b89-926b-7e7d86e9cb4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:26.621279 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:26.621392 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144e6a19-b618-4b89-926b-7e7d86e9cb4c None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:26.621628 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144e6a19-b618-4b89-926b-7e7d86e9cb4c None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:26.621944 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144e6a19-b618-4b89-926b-7e7d86e9cb4c None admin] PATH_INFO: `/v3/domains` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:26.622692 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144e6a19-b618-4b89-926b-7e7d86e9cb4c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:04:26.622899 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144e6a19-b618-4b89-926b-7e7d86e9cb4c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:26.624228 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144e6a19-b618-4b89-926b-7e7d86e9cb4c None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:26.640508 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 306/591] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 31 01:04:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:04:26.640508 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144e6a19-b618-4b89-926b-7e7d86e9cb4c None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:26.642112 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1982ba9a-763c-4853-8301-7e07a6d058c6 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:26.678876 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f29693-6638-448e-bb91-228c251dc374 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:26.680136 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f29693-6638-448e-bb91-228c251dc374 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:26.680430 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f29693-6638-448e-bb91-228c251dc374 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:26.680697 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f29693-6638-448e-bb91-228c251dc374 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:26.691821 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f29693-6638-448e-bb91-228c251dc374 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:26.693307 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f29693-6638-448e-bb91-228c251dc374 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:26.714226 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application [None req-58f29693-6638-448e-bb91-228c251dc374 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 01:04:26.714226 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:04:26.714226 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:04:26.714226 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:04:26.714226 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:04:26.714226 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:04:26.714226 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:04:26.714226 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:04:26.714226 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:04:26.714226 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:04:26.714226 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:04:26.714226 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:04:26.714226 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 01:04:26.714226 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 01:04:26.714226 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 01:04:26.714226 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 01:04:26.715239 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:04:26.715239 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:04:26.715239 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:04:26.715239 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:04:26.715239 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:04:26.715239 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:04:26.715239 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:04:26.715239 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application with Lock( Aug 31 01:04:26.715239 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:04:26.715239 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:04:26.715239 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:04:26.715239 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:04:26.715239 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:04:26.715239 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:04:26.715239 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:04:26.715239 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:04:26.715239 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 01:04:26.716115 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 01:04:26.716115 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 01:04:26.716115 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Aug 31 01:04:26.718845 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 286/592] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 01:04:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:04:26.720101 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f29693-6638-448e-bb91-228c251dc374 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:26.732543 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:26.769379 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144e6a19-b618-4b89-926b-7e7d86e9cb4c None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:26.805993 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fefa79d-33e3-4159-af0a-9994bdb21e5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:26.806743 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fefa79d-33e3-4159-af0a-9994bdb21e5d None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:26.807116 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fefa79d-33e3-4159-af0a-9994bdb21e5d None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:26.807523 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fefa79d-33e3-4159-af0a-9994bdb21e5d None admin] PATH_INFO: `/v3/domains` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:26.808273 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fefa79d-33e3-4159-af0a-9994bdb21e5d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:04:26.808672 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fefa79d-33e3-4159-af0a-9994bdb21e5d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:26.810305 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fefa79d-33e3-4159-af0a-9994bdb21e5d None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:26.823274 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 307/593] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 31 01:04:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:04:26.824586 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fefa79d-33e3-4159-af0a-9994bdb21e5d None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:26.850965 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f29693-6638-448e-bb91-228c251dc374 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:26.885999 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c96571e-6d24-4ae7-9312-457a37f419c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:26.886639 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c96571e-6d24-4ae7-9312-457a37f419c1 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:26.886931 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c96571e-6d24-4ae7-9312-457a37f419c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:26.887209 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c96571e-6d24-4ae7-9312-457a37f419c1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:26.896783 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c96571e-6d24-4ae7-9312-457a37f419c1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:26.904755 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c96571e-6d24-4ae7-9312-457a37f419c1 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:26.910113 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application [None req-2c96571e-6d24-4ae7-9312-457a37f419c1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 01:04:26.910113 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:04:26.910113 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:04:26.910113 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:04:26.910113 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:04:26.910113 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:04:26.910113 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:04:26.910113 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:04:26.910113 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:04:26.910113 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:04:26.910113 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:04:26.910113 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:04:26.910113 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 01:04:26.910113 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 01:04:26.910113 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 01:04:26.910975 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 01:04:26.910975 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:04:26.910975 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:04:26.910975 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:04:26.910975 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:04:26.910975 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:04:26.910975 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:04:26.910975 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:04:26.910975 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application with Lock( Aug 31 01:04:26.910975 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:04:26.910975 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:04:26.910975 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:04:26.910975 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:04:26.910975 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:04:26.910975 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:04:26.910975 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:04:26.910975 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:04:26.911857 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 01:04:26.911857 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 01:04:26.911857 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 01:04:26.911857 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 01:04:26.911857 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 01:04:26.911857 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 01:04:26.911857 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 01:04:26.911857 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Aug 31 01:04:26.916124 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:26.916511 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 287/594] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Aug 31 01:04:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:04:26.917391 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c96571e-6d24-4ae7-9312-457a37f419c1 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:26.951503 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fefa79d-33e3-4159-af0a-9994bdb21e5d None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:26.982521 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:26.997376 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c367a916-3fd4-49a9-84aa-736d5d594eed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:26.998280 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c367a916-3fd4-49a9-84aa-736d5d594eed None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:26.999105 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c367a916-3fd4-49a9-84aa-736d5d594eed None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:26.999559 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c367a916-3fd4-49a9-84aa-736d5d594eed None admin] PATH_INFO: `/v3/domains` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:27.000340 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c367a916-3fd4-49a9-84aa-736d5d594eed None admin] RBAC: Adding query filter params (name=Default) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:04:27.001176 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c367a916-3fd4-49a9-84aa-736d5d594eed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:27.003306 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c367a916-3fd4-49a9-84aa-736d5d594eed None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:27.023791 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 308/595] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 31 01:04:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:04:27.023791 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c367a916-3fd4-49a9-84aa-736d5d594eed None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:27.032614 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c96571e-6d24-4ae7-9312-457a37f419c1 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:27.067895 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb22cbb-d77a-4a80-b01d-36b00bc6d93d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:27.067895 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb22cbb-d77a-4a80-b01d-36b00bc6d93d None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:27.068211 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb22cbb-d77a-4a80-b01d-36b00bc6d93d None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:27.068449 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb22cbb-d77a-4a80-b01d-36b00bc6d93d None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:27.076324 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb22cbb-d77a-4a80-b01d-36b00bc6d93d None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:27.077825 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb22cbb-d77a-4a80-b01d-36b00bc6d93d None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:27.089726 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application [None req-6bb22cbb-d77a-4a80-b01d-36b00bc6d93d None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 31 01:04:27.089726 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:04:27.089726 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:04:27.089726 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:04:27.089726 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:04:27.089726 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:04:27.089726 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:04:27.089726 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:04:27.089726 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:04:27.089726 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:04:27.089726 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:04:27.089726 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:04:27.089726 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 01:04:27.089726 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 01:04:27.089726 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 01:04:27.089726 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 01:04:27.090667 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:04:27.090667 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:04:27.090667 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 01:04:27.090667 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 01:04:27.090667 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 01:04:27.090667 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 01:04:27.090667 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:04:27.090667 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:04:27.090667 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:04:27.090667 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:04:27.090667 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:04:27.090667 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application with Lock( Aug 31 01:04:27.090667 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:04:27.090667 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:04:27.090667 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:04:27.090667 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:04:27.090667 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:04:27.090667 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:04:27.091594 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:04:27.091594 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:04:27.091594 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 01:04:27.091594 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 01:04:27.091594 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 01:04:27.091594 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 01:04:27.091594 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 01:04:27.091594 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 01:04:27.091594 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 31 01:04:27.091594 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Aug 31 01:04:27.095602 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 288/596] 10.222.0.20 () {56 vars in 1102 bytes} [Wed Aug 31 01:04:26 2022] GET /identity/v3/users/neutron => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:04:27.096453 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb22cbb-d77a-4a80-b01d-36b00bc6d93d None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:27.143774 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c367a916-3fd4-49a9-84aa-736d5d594eed None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:27.158159 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:27.180314 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719ea1af-0e1b-41fb-a028-6a4b43a3fe95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:27.180966 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719ea1af-0e1b-41fb-a028-6a4b43a3fe95 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:27.181287 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719ea1af-0e1b-41fb-a028-6a4b43a3fe95 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:27.181562 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719ea1af-0e1b-41fb-a028-6a4b43a3fe95 None admin] PATH_INFO: `/v3/projects` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:27.182250 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719ea1af-0e1b-41fb-a028-6a4b43a3fe95 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:04:27.182561 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719ea1af-0e1b-41fb-a028-6a4b43a3fe95 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:27.183980 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719ea1af-0e1b-41fb-a028-6a4b43a3fe95 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:27.195766 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 309/597] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Aug 31 01:04:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:04:27.196574 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719ea1af-0e1b-41fb-a028-6a4b43a3fe95 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:27.208632 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb22cbb-d77a-4a80-b01d-36b00bc6d93d None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:27.246178 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:27.246969 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7ccc46-62cd-4026-a32d-d7ddda6a11a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:27.247603 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7ccc46-62cd-4026-a32d-d7ddda6a11a4 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:27.247892 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7ccc46-62cd-4026-a32d-d7ddda6a11a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:27.248168 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7ccc46-62cd-4026-a32d-d7ddda6a11a4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:27.254989 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7ccc46-62cd-4026-a32d-d7ddda6a11a4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:27.256444 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7ccc46-62cd-4026-a32d-d7ddda6a11a4 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:27.268176 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application [None req-3f7ccc46-62cd-4026-a32d-d7ddda6a11a4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 01:04:27.268176 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:04:27.268176 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:04:27.268176 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:04:27.268176 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:04:27.268176 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:04:27.268176 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:04:27.268176 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:04:27.268176 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:04:27.268176 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:04:27.268176 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:04:27.268176 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:04:27.268176 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 01:04:27.268176 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 01:04:27.268176 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 01:04:27.269203 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 01:04:27.269203 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:04:27.269203 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:04:27.269203 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:04:27.269203 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:04:27.269203 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:04:27.269203 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:04:27.269203 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:04:27.269203 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application with Lock( Aug 31 01:04:27.269203 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:04:27.269203 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:04:27.269203 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:04:27.269203 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:04:27.269203 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:04:27.269203 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:04:27.269203 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:04:27.269203 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:04:27.270084 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 01:04:27.270084 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 01:04:27.270084 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 01:04:27.270084 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 01:04:27.270084 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 01:04:27.270084 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 01:04:27.270084 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 01:04:27.270084 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Aug 31 01:04:27.270084 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 289/598] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Aug 31 01:04:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:04:27.270564 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db4bb6f-957c-4b15-8ba5-2707252bcc85 None None] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:27.270958 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db4bb6f-957c-4b15-8ba5-2707252bcc85 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:27.271062 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db4bb6f-957c-4b15-8ba5-2707252bcc85 None None] PATH_INFO: `/` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:27.272405 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 290/599] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:04:27 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:04:27.277516 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db4bb6f-957c-4b15-8ba5-2707252bcc85 None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:27.311797 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719ea1af-0e1b-41fb-a028-6a4b43a3fe95 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:27.347051 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f85d6e-22b8-4f64-8f85-581506b33146 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:27.347683 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f85d6e-22b8-4f64-8f85-581506b33146 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:27.347964 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f85d6e-22b8-4f64-8f85-581506b33146 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:27.348244 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f85d6e-22b8-4f64-8f85-581506b33146 None admin] PATH_INFO: `/v3/users` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:27.348997 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f85d6e-22b8-4f64-8f85-581506b33146 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:04:27.349304 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f85d6e-22b8-4f64-8f85-581506b33146 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:27.350727 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f85d6e-22b8-4f64-8f85-581506b33146 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:27.369849 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 310/600] 10.222.0.20 () {56 vars in 1139 bytes} [Wed Aug 31 01:04:27 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:04:27.370677 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f85d6e-22b8-4f64-8f85-581506b33146 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:27.402239 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db4bb6f-957c-4b15-8ba5-2707252bcc85 None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:27.424628 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:27.440103 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64f2f9d-b5bc-4427-914a-2d23b454e4e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:27.440782 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64f2f9d-b5bc-4427-914a-2d23b454e4e1 None admin] REQUEST_METHOD: `PUT` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:27.441072 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64f2f9d-b5bc-4427-914a-2d23b454e4e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:27.441340 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64f2f9d-b5bc-4427-914a-2d23b454e4e1 None admin] PATH_INFO: `/v3/projects/60b2abc15c2849ba8a1ed92117cc981b/users/d8590c288028418ebe7a4163fd8039f1/roles/7d00a4cfa13744a69604ac2cf1a5816e` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:27.456388 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64f2f9d-b5bc-4427-914a-2d23b454e4e1 None admin] RBAC: Authorizing `identity:create_grant(project_id=60b2abc15c2849ba8a1ed92117cc981b, user_id=d8590c288028418ebe7a4163fd8039f1, role_id=7d00a4cfa13744a69604ac2cf1a5816e)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:27.458607 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64f2f9d-b5bc-4427-914a-2d23b454e4e1 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:27.481854 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:27.484759 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f85d6e-22b8-4f64-8f85-581506b33146 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:27.518447 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc7955b-c67a-4308-b76c-d7c6560ae588 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:27.519050 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc7955b-c67a-4308-b76c-d7c6560ae588 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:27.519331 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc7955b-c67a-4308-b76c-d7c6560ae588 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:27.519599 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc7955b-c67a-4308-b76c-d7c6560ae588 None admin] PATH_INFO: `/v3/projects` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:27.520284 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc7955b-c67a-4308-b76c-d7c6560ae588 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:04:27.520590 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc7955b-c67a-4308-b76c-d7c6560ae588 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:27.522051 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc7955b-c67a-4308-b76c-d7c6560ae588 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:27.526873 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 291/601] 10.222.0.20 () {58 vars in 1439 bytes} [Wed Aug 31 01:04:27 2022] PUT /identity/v3/projects/60b2abc15c2849ba8a1ed92117cc981b/users/d8590c288028418ebe7a4163fd8039f1/roles/7d00a4cfa13744a69604ac2cf1a5816e => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:04:27.539171 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 311/602] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Aug 31 01:04:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:04:27.539171 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc7955b-c67a-4308-b76c-d7c6560ae588 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:27.539451 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a7d015-a9a7-4709-8b5e-8684422195af None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:27.539451 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a7d015-a9a7-4709-8b5e-8684422195af None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:27.539451 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a7d015-a9a7-4709-8b5e-8684422195af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:27.586526 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-51a7d015-a9a7-4709-8b5e-8684422195af None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:04:27.588111 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-51a7d015-a9a7-4709-8b5e-8684422195af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:04:27.672998 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc7955b-c67a-4308-b76c-d7c6560ae588 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:27.684533 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 292/603] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:04:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 01:04:27.685662 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a7d015-a9a7-4709-8b5e-8684422195af None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:27.720440 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901234ad-6702-4694-8c64-396931508a4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:27.721804 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901234ad-6702-4694-8c64-396931508a4f None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:27.722089 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901234ad-6702-4694-8c64-396931508a4f None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:27.722362 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901234ad-6702-4694-8c64-396931508a4f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:27.736858 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901234ad-6702-4694-8c64-396931508a4f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:27.738871 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901234ad-6702-4694-8c64-396931508a4f None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:27.757441 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application [None req-901234ad-6702-4694-8c64-396931508a4f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 01:04:27.757441 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:04:27.757441 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:04:27.757441 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:04:27.757441 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:04:27.757441 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:04:27.757441 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:04:27.757441 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:04:27.757441 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:04:27.757441 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:04:27.757441 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:04:27.757441 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:04:27.757441 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 01:04:27.757441 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 01:04:27.757441 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 01:04:27.757441 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 01:04:27.758984 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:04:27.758984 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:04:27.758984 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:04:27.758984 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:04:27.758984 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:04:27.758984 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:04:27.758984 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:04:27.758984 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application with Lock( Aug 31 01:04:27.758984 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:04:27.758984 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:04:27.758984 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:04:27.758984 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:04:27.758984 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:04:27.758984 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:04:27.758984 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:04:27.758984 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:04:27.758984 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 01:04:27.759870 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 01:04:27.759870 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 01:04:27.759870 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Aug 31 01:04:27.760636 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 312/604] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 01:04:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:04:27.766368 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901234ad-6702-4694-8c64-396931508a4f None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:27.787194 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:27.810007 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:27.813162 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a7d015-a9a7-4709-8b5e-8684422195af None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:27.852366 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ef3abf-5e89-4902-b9b6-eef5398f8c71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:27.853119 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ef3abf-5e89-4902-b9b6-eef5398f8c71 None admin] REQUEST_METHOD: `PUT` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:27.853404 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ef3abf-5e89-4902-b9b6-eef5398f8c71 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:27.853735 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ef3abf-5e89-4902-b9b6-eef5398f8c71 None admin] PATH_INFO: `/v3/projects/60b2abc15c2849ba8a1ed92117cc981b/users/a5c8fc69c518412ba1aad61a9f4f558c/roles/7d00a4cfa13744a69604ac2cf1a5816e` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:27.872294 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ef3abf-5e89-4902-b9b6-eef5398f8c71 None admin] RBAC: Authorizing `identity:create_grant(project_id=60b2abc15c2849ba8a1ed92117cc981b, user_id=a5c8fc69c518412ba1aad61a9f4f558c, role_id=7d00a4cfa13744a69604ac2cf1a5816e)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:27.873710 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ef3abf-5e89-4902-b9b6-eef5398f8c71 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:27.898737 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901234ad-6702-4694-8c64-396931508a4f None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:27.935380 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7046dc13-616f-44a7-8a1f-c61650a0abeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:27.935919 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7046dc13-616f-44a7-8a1f-c61650a0abeb None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:27.936201 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7046dc13-616f-44a7-8a1f-c61650a0abeb None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:27.936472 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7046dc13-616f-44a7-8a1f-c61650a0abeb None admin] PATH_INFO: `/v3/domains` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:27.937151 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7046dc13-616f-44a7-8a1f-c61650a0abeb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:04:27.937455 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7046dc13-616f-44a7-8a1f-c61650a0abeb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:27.938818 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7046dc13-616f-44a7-8a1f-c61650a0abeb None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:27.952805 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 313/605] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 31 01:04:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:04:27.958770 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7046dc13-616f-44a7-8a1f-c61650a0abeb None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:27.977967 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 293/606] 10.222.0.20 () {58 vars in 1439 bytes} [Wed Aug 31 01:04:27 2022] PUT /identity/v3/projects/60b2abc15c2849ba8a1ed92117cc981b/users/a5c8fc69c518412ba1aad61a9f4f558c/roles/7d00a4cfa13744a69604ac2cf1a5816e => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:04:28.054173 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:28.096446 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ef3abf-5e89-4902-b9b6-eef5398f8c71 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:28.097064 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7046dc13-616f-44a7-8a1f-c61650a0abeb None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:28.139219 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a711a07-84ac-412e-bd48-f12fdf3f182a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:28.139867 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a711a07-84ac-412e-bd48-f12fdf3f182a None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:28.140839 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a711a07-84ac-412e-bd48-f12fdf3f182a None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:28.140839 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a711a07-84ac-412e-bd48-f12fdf3f182a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:28.148566 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a711a07-84ac-412e-bd48-f12fdf3f182a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:28.150569 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a711a07-84ac-412e-bd48-f12fdf3f182a None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:28.163298 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application [None req-8a711a07-84ac-412e-bd48-f12fdf3f182a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 01:04:28.163298 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:04:28.163298 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:04:28.163298 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:04:28.163298 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:04:28.163298 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:04:28.163298 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:04:28.163298 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:04:28.163298 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:04:28.163298 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:04:28.163298 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:04:28.163298 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:04:28.163298 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 01:04:28.163298 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 01:04:28.163298 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 01:04:28.164299 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 01:04:28.164299 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:04:28.164299 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:04:28.164299 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:04:28.164299 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:04:28.164299 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:04:28.164299 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:04:28.164299 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:04:28.164299 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application with Lock( Aug 31 01:04:28.164299 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:04:28.164299 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:04:28.164299 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:04:28.164299 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:04:28.164299 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:04:28.164299 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:04:28.164299 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:04:28.164299 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:04:28.165915 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 01:04:28.165915 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 01:04:28.165915 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 01:04:28.165915 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 01:04:28.165915 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 01:04:28.165915 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 01:04:28.165915 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 01:04:28.165915 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Aug 31 01:04:28.165915 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 314/607] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Aug 31 01:04:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:04:28.170311 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a711a07-84ac-412e-bd48-f12fdf3f182a None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:28.278640 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ef3abf-5e89-4902-b9b6-eef5398f8c71 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:28.317034 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61df47b7-ceca-44e0-9cf5-fdfe1ac8cd6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:28.317724 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61df47b7-ceca-44e0-9cf5-fdfe1ac8cd6f None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:28.318006 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61df47b7-ceca-44e0-9cf5-fdfe1ac8cd6f None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:28.318376 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61df47b7-ceca-44e0-9cf5-fdfe1ac8cd6f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:28.330096 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61df47b7-ceca-44e0-9cf5-fdfe1ac8cd6f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 01:04:28.334636 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a711a07-84ac-412e-bd48-f12fdf3f182a None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:28.335032 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application [None req-61df47b7-ceca-44e0-9cf5-fdfe1ac8cd6f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 01:04:28.335032 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:04:28.335032 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:04:28.335032 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:04:28.335032 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:04:28.335032 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:04:28.335032 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:04:28.335032 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:04:28.335032 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:04:28.335032 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:04:28.335032 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:04:28.335032 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:04:28.335032 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 01:04:28.335032 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 01:04:28.335032 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 01:04:28.335032 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 01:04:28.337776 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 01:04:28.337776 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 01:04:28.337776 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 01:04:28.337776 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 01:04:28.337776 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:04:28.337776 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:04:28.337776 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:04:28.337776 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:04:28.337776 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:04:28.337776 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:04:28.337776 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:04:28.337776 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application with Lock( Aug 31 01:04:28.337776 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:04:28.337776 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:04:28.337776 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:04:28.337776 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:04:28.339777 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:04:28.339777 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:04:28.339777 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:04:28.339777 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:04:28.339777 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 01:04:28.339777 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 01:04:28.339777 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 01:04:28.339777 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 01:04:28.339777 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 01:04:28.339777 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 01:04:28.339777 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 01:04:28.339777 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Aug 31 01:04:28.339777 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 294/608] 10.222.0.20 () {56 vars in 1102 bytes} [Wed Aug 31 01:04:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 01:04:28.349522 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61df47b7-ceca-44e0-9cf5-fdfe1ac8cd6f None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:28.369469 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:28.390763 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:28.400565 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84b75c6-aeee-4f6a-bbdf-08e5f9b8f81f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:28.401743 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84b75c6-aeee-4f6a-bbdf-08e5f9b8f81f None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:28.402222 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84b75c6-aeee-4f6a-bbdf-08e5f9b8f81f None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:28.402693 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84b75c6-aeee-4f6a-bbdf-08e5f9b8f81f None admin] PATH_INFO: `/v3/projects` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:28.404258 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84b75c6-aeee-4f6a-bbdf-08e5f9b8f81f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:04:28.404968 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84b75c6-aeee-4f6a-bbdf-08e5f9b8f81f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:28.407688 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84b75c6-aeee-4f6a-bbdf-08e5f9b8f81f None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:28.432550 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 315/609] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Aug 31 01:04:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 01:04:28.434785 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a66f8c0-a104-4e45-acce-60f23366c57d None None] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:28.435265 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a66f8c0-a104-4e45-acce-60f23366c57d None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:28.435909 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a66f8c0-a104-4e45-acce-60f23366c57d None None] PATH_INFO: `/` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:28.443907 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 316/610] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:04:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:04:28.443907 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a66f8c0-a104-4e45-acce-60f23366c57d None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:28.521772 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61df47b7-ceca-44e0-9cf5-fdfe1ac8cd6f None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:28.558906 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af74aceb-e764-482e-921e-0cfd0a6f6de5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:28.560580 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af74aceb-e764-482e-921e-0cfd0a6f6de5 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:28.561124 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af74aceb-e764-482e-921e-0cfd0a6f6de5 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:28.561833 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af74aceb-e764-482e-921e-0cfd0a6f6de5 None admin] PATH_INFO: `/v3/roles` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:28.563218 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af74aceb-e764-482e-921e-0cfd0a6f6de5 None admin] RBAC: Adding query filter params (name=service) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:04:28.563838 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af74aceb-e764-482e-921e-0cfd0a6f6de5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:28.565472 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af74aceb-e764-482e-921e-0cfd0a6f6de5 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:28.566366 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a66f8c0-a104-4e45-acce-60f23366c57d None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:28.577481 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 295/611] 10.222.0.20 () {56 vars in 1103 bytes} [Wed Aug 31 01:04:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:04:28.579603 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446139ba-93ab-4bfc-9be7-20cc680dea0e None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:28.580482 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446139ba-93ab-4bfc-9be7-20cc680dea0e None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:28.581240 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446139ba-93ab-4bfc-9be7-20cc680dea0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:28.604948 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-446139ba-93ab-4bfc-9be7-20cc680dea0e None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:04:28.606204 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3f86b2-9452-48a7-b900-5085f02e9649 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:28.606698 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-446139ba-93ab-4bfc-9be7-20cc680dea0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:04:28.606873 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3f86b2-9452-48a7-b900-5085f02e9649 None admin] REQUEST_METHOD: `PUT` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:28.607188 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3f86b2-9452-48a7-b900-5085f02e9649 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:28.607463 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3f86b2-9452-48a7-b900-5085f02e9649 None admin] PATH_INFO: `/v3/projects/60b2abc15c2849ba8a1ed92117cc981b/users/0817c3320d5d41d8bba0550c8ae8dd60/roles/7d00a4cfa13744a69604ac2cf1a5816e` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:28.626890 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3f86b2-9452-48a7-b900-5085f02e9649 None admin] RBAC: Authorizing `identity:create_grant(project_id=60b2abc15c2849ba8a1ed92117cc981b, user_id=0817c3320d5d41d8bba0550c8ae8dd60, role_id=7d00a4cfa13744a69604ac2cf1a5816e)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:28.628946 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3f86b2-9452-48a7-b900-5085f02e9649 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:28.688749 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 296/612] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:04:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 01:04:28.688749 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446139ba-93ab-4bfc-9be7-20cc680dea0e None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:28.691345 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:28.710092 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 317/613] 10.222.0.20 () {58 vars in 1439 bytes} [Wed Aug 31 01:04:28 2022] PUT /identity/v3/projects/60b2abc15c2849ba8a1ed92117cc981b/users/0817c3320d5d41d8bba0550c8ae8dd60/roles/7d00a4cfa13744a69604ac2cf1a5816e => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:04:28.811010 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446139ba-93ab-4bfc-9be7-20cc680dea0e None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:28.861696 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ae4adf-769a-428b-9499-add5f7a31075 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:28.862334 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ae4adf-769a-428b-9499-add5f7a31075 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:28.862615 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ae4adf-769a-428b-9499-add5f7a31075 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:28.862895 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ae4adf-769a-428b-9499-add5f7a31075 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:28.876972 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ae4adf-769a-428b-9499-add5f7a31075 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:28.878468 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ae4adf-769a-428b-9499-add5f7a31075 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:28.913180 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application [None req-80ae4adf-769a-428b-9499-add5f7a31075 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 01:04:28.913180 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:04:28.913180 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:04:28.913180 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:04:28.913180 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:04:28.913180 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:04:28.913180 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:04:28.913180 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:04:28.913180 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:04:28.913180 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:04:28.913180 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:04:28.913180 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:04:28.913180 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 01:04:28.913180 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 01:04:28.913180 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 01:04:28.913180 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 01:04:28.914689 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:04:28.914689 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:04:28.914689 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:04:28.914689 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:04:28.914689 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:04:28.914689 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:04:28.914689 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:04:28.914689 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application with Lock( Aug 31 01:04:28.914689 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:04:28.914689 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:04:28.914689 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:04:28.914689 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:04:28.914689 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:04:28.914689 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:04:28.914689 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:04:28.914689 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:04:28.914689 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 01:04:28.915950 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 01:04:28.915950 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 01:04:28.915950 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Aug 31 01:04:28.915950 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 297/614] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 01:04:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:04:28.920967 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3f86b2-9452-48a7-b900-5085f02e9649 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:28.971246 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:29.033586 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3f86b2-9452-48a7-b900-5085f02e9649 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:29.077797 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08399648-e5dc-4961-9aeb-a1a034c351df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:29.078334 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08399648-e5dc-4961-9aeb-a1a034c351df None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:29.078620 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08399648-e5dc-4961-9aeb-a1a034c351df None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:29.078899 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08399648-e5dc-4961-9aeb-a1a034c351df None admin] PATH_INFO: `/v3/domains` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:29.079558 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08399648-e5dc-4961-9aeb-a1a034c351df None admin] RBAC: Adding query filter params (name=Default) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:04:29.079870 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08399648-e5dc-4961-9aeb-a1a034c351df None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:29.081280 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08399648-e5dc-4961-9aeb-a1a034c351df None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:29.087546 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ae4adf-769a-428b-9499-add5f7a31075 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:29.104932 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 318/615] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 31 01:04:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:04:29.111724 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08399648-e5dc-4961-9aeb-a1a034c351df None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:29.202226 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:29.292851 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ae4adf-769a-428b-9499-add5f7a31075 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:29.304027 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08399648-e5dc-4961-9aeb-a1a034c351df None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:29.305793 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:29.342364 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e8047c-3570-47bf-8f45-29da3a24e907 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:29.342980 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e8047c-3570-47bf-8f45-29da3a24e907 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:29.343264 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e8047c-3570-47bf-8f45-29da3a24e907 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:29.343545 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e8047c-3570-47bf-8f45-29da3a24e907 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:29.352226 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e8047c-3570-47bf-8f45-29da3a24e907 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:29.353831 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e8047c-3570-47bf-8f45-29da3a24e907 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:29.365942 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application [None req-a8e8047c-3570-47bf-8f45-29da3a24e907 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 31 01:04:29.365942 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:04:29.365942 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:04:29.365942 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:04:29.365942 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:04:29.365942 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:04:29.365942 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:04:29.365942 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:04:29.365942 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:04:29.365942 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:04:29.365942 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:04:29.365942 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:04:29.365942 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 01:04:29.365942 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 01:04:29.365942 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 01:04:29.365942 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 01:04:29.366969 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:04:29.366969 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:04:29.366969 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 01:04:29.366969 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 01:04:29.366969 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 01:04:29.366969 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 01:04:29.366969 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:04:29.366969 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:04:29.366969 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:04:29.366969 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:04:29.366969 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:04:29.366969 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application with Lock( Aug 31 01:04:29.366969 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:04:29.366969 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:04:29.366969 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:04:29.366969 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:04:29.366969 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:04:29.366969 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:04:29.368603 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:04:29.368603 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:04:29.368603 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 01:04:29.368603 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 01:04:29.368603 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 01:04:29.368603 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 01:04:29.368603 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 01:04:29.368603 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 01:04:29.368603 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 31 01:04:29.368603 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Aug 31 01:04:29.368603 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 319/616] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Aug 31 01:04:29 2022] GET /identity/v3/users/swift => generated 83 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:04:29.369235 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c0868b-fb1d-422f-bce4-371a074baf2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:29.370638 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c0868b-fb1d-422f-bce4-371a074baf2f None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:29.370638 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c0868b-fb1d-422f-bce4-371a074baf2f None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:29.370638 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c0868b-fb1d-422f-bce4-371a074baf2f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:29.373272 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e8047c-3570-47bf-8f45-29da3a24e907 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:29.399182 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c0868b-fb1d-422f-bce4-371a074baf2f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 01:04:29.405142 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application [None req-71c0868b-fb1d-422f-bce4-371a074baf2f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 01:04:29.405142 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:04:29.405142 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:04:29.405142 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:04:29.405142 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:04:29.405142 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:04:29.405142 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:04:29.405142 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:04:29.405142 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:04:29.405142 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:04:29.405142 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:04:29.405142 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:04:29.405142 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 01:04:29.405142 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 01:04:29.405142 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 01:04:29.405142 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 01:04:29.406064 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 01:04:29.406064 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 01:04:29.406064 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 01:04:29.406064 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 01:04:29.406064 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:04:29.406064 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:04:29.406064 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:04:29.406064 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:04:29.406064 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:04:29.406064 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:04:29.406064 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:04:29.406064 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application with Lock( Aug 31 01:04:29.406064 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:04:29.406064 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:04:29.406064 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:04:29.406064 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:04:29.406909 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:04:29.406909 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:04:29.406909 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:04:29.406909 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:04:29.406909 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 01:04:29.406909 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 01:04:29.406909 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 01:04:29.406909 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 01:04:29.406909 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 01:04:29.406909 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 01:04:29.406909 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 01:04:29.406909 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Aug 31 01:04:29.413847 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 298/617] 10.222.0.20 () {56 vars in 1102 bytes} [Wed Aug 31 01:04:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 327 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:04:29.420414 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c0868b-fb1d-422f-bce4-371a074baf2f None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:29.481927 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e8047c-3570-47bf-8f45-29da3a24e907 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:29.547945 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cea1e73-ec96-4239-a9cc-d1bde2efb86b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:29.548560 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cea1e73-ec96-4239-a9cc-d1bde2efb86b None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:29.548874 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cea1e73-ec96-4239-a9cc-d1bde2efb86b None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:29.549154 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cea1e73-ec96-4239-a9cc-d1bde2efb86b None admin] PATH_INFO: `/v3/users` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:29.549882 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cea1e73-ec96-4239-a9cc-d1bde2efb86b None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:04:29.550189 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cea1e73-ec96-4239-a9cc-d1bde2efb86b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:29.551625 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cea1e73-ec96-4239-a9cc-d1bde2efb86b None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:29.587168 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 320/618] 10.222.0.20 () {56 vars in 1135 bytes} [Wed Aug 31 01:04:29 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 01:04:29.588324 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cea1e73-ec96-4239-a9cc-d1bde2efb86b None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:29.605951 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c0868b-fb1d-422f-bce4-371a074baf2f None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:29.621089 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:29.642669 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f4a5de-1193-4878-9776-45143402d0a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:29.643575 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f4a5de-1193-4878-9776-45143402d0a8 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:29.644128 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f4a5de-1193-4878-9776-45143402d0a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:29.644647 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f4a5de-1193-4878-9776-45143402d0a8 None admin] PATH_INFO: `/v3/roles` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:29.645605 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f4a5de-1193-4878-9776-45143402d0a8 None admin] RBAC: Adding query filter params (name=service) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:04:29.646117 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f4a5de-1193-4878-9776-45143402d0a8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:29.647699 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f4a5de-1193-4878-9776-45143402d0a8 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:29.650717 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:29.658076 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 299/619] 10.222.0.20 () {56 vars in 1103 bytes} [Wed Aug 31 01:04:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 01:04:29.667329 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f4a5de-1193-4878-9776-45143402d0a8 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:29.782302 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cea1e73-ec96-4239-a9cc-d1bde2efb86b None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:29.823823 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b557b3e-95ec-4dee-ab23-8fcdadf424f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:29.824744 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b557b3e-95ec-4dee-ab23-8fcdadf424f9 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:29.825040 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b557b3e-95ec-4dee-ab23-8fcdadf424f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:29.825307 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b557b3e-95ec-4dee-ab23-8fcdadf424f9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:29.845687 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b557b3e-95ec-4dee-ab23-8fcdadf424f9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:29.849204 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b557b3e-95ec-4dee-ab23-8fcdadf424f9 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:29.853403 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f4a5de-1193-4878-9776-45143402d0a8 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:29.876770 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application [None req-0b557b3e-95ec-4dee-ab23-8fcdadf424f9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 01:04:29.876770 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:04:29.876770 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:04:29.876770 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:04:29.876770 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:04:29.876770 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:04:29.876770 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:04:29.876770 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:04:29.876770 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:04:29.876770 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:04:29.876770 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:04:29.876770 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:04:29.876770 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 01:04:29.876770 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 01:04:29.876770 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 01:04:29.876770 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 01:04:29.877782 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:04:29.877782 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:04:29.877782 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:04:29.877782 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:04:29.877782 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:04:29.877782 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:04:29.877782 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:04:29.877782 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application with Lock( Aug 31 01:04:29.877782 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:04:29.877782 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:04:29.877782 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:04:29.877782 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:04:29.877782 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:04:29.877782 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:04:29.877782 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:04:29.877782 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:04:29.877782 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 01:04:29.879229 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 01:04:29.879229 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 01:04:29.879229 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Aug 31 01:04:29.879229 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 321/620] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 01:04:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 01:04:29.884150 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b557b3e-95ec-4dee-ab23-8fcdadf424f9 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:29.934700 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf42afe3-1765-47c3-bdec-e8f04a436f15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:29.935505 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf42afe3-1765-47c3-bdec-e8f04a436f15 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:29.935863 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf42afe3-1765-47c3-bdec-e8f04a436f15 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:29.936251 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf42afe3-1765-47c3-bdec-e8f04a436f15 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:29.947142 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf42afe3-1765-47c3-bdec-e8f04a436f15 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:29.948766 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf42afe3-1765-47c3-bdec-e8f04a436f15 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:29.965760 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application [None req-bf42afe3-1765-47c3-bdec-e8f04a436f15 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 01:04:29.965760 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:04:29.965760 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:04:29.965760 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:04:29.965760 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:04:29.965760 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:04:29.965760 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:04:29.965760 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:04:29.965760 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:04:29.965760 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:04:29.965760 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:04:29.965760 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:04:29.965760 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 01:04:29.965760 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 01:04:29.965760 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 01:04:29.965760 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 01:04:29.966727 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:04:29.966727 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:04:29.966727 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:04:29.966727 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:04:29.966727 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:04:29.966727 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:04:29.966727 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:04:29.966727 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application with Lock( Aug 31 01:04:29.966727 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:04:29.966727 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:04:29.966727 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:04:29.966727 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:04:29.966727 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:04:29.966727 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:04:29.966727 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:04:29.966727 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:04:29.966727 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 01:04:29.967613 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 01:04:29.967613 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 01:04:29.967613 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Aug 31 01:04:29.967613 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 300/621] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 01:04:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:04:29.969225 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997f5419-04b9-48f4-bca7-809826a45481 None None] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:29.969596 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997f5419-04b9-48f4-bca7-809826a45481 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:29.969943 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997f5419-04b9-48f4-bca7-809826a45481 None None] PATH_INFO: `/` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:29.975568 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 301/622] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:04:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:04:29.977088 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997f5419-04b9-48f4-bca7-809826a45481 None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:29.995790 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:30.007344 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:30.031459 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b557b3e-95ec-4dee-ab23-8fcdadf424f9 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:30.070337 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55d7660-eda7-4add-86ee-535b9888349d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:30.071283 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55d7660-eda7-4add-86ee-535b9888349d None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:30.071743 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55d7660-eda7-4add-86ee-535b9888349d None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:30.072194 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55d7660-eda7-4add-86ee-535b9888349d None admin] PATH_INFO: `/v3/domains` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:30.073078 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55d7660-eda7-4add-86ee-535b9888349d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:04:30.073574 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55d7660-eda7-4add-86ee-535b9888349d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:30.075373 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55d7660-eda7-4add-86ee-535b9888349d None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:30.088472 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 322/623] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 31 01:04:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:04:30.090321 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6905add-d249-4913-a129-e39730fa3b04 None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:30.090807 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6905add-d249-4913-a129-e39730fa3b04 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:30.090915 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997f5419-04b9-48f4-bca7-809826a45481 None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:30.091355 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6905add-d249-4913-a129-e39730fa3b04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:30.109746 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-a6905add-d249-4913-a129-e39730fa3b04 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:04:30.111192 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-a6905add-d249-4913-a129-e39730fa3b04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:04:30.128926 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6216ed94-d5e7-4451-ae85-d29932f4259f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:30.129823 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6216ed94-d5e7-4451-ae85-d29932f4259f None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:30.130099 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6216ed94-d5e7-4451-ae85-d29932f4259f None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:30.130450 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6216ed94-d5e7-4451-ae85-d29932f4259f None admin] PATH_INFO: `/v3/domains` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:30.131210 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6216ed94-d5e7-4451-ae85-d29932f4259f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:04:30.131587 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6216ed94-d5e7-4451-ae85-d29932f4259f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:30.133201 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6216ed94-d5e7-4451-ae85-d29932f4259f None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:30.147057 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 302/624] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 31 01:04:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:04:30.148651 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6216ed94-d5e7-4451-ae85-d29932f4259f None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:30.181978 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 323/625] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:04:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 01:04:30.183167 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6905add-d249-4913-a129-e39730fa3b04 None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:30.244856 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:30.283599 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6905add-d249-4913-a129-e39730fa3b04 None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:30.318805 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a09064-5238-4ad0-bc87-ea6951d4c084 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:30.319782 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a09064-5238-4ad0-bc87-ea6951d4c084 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:30.320401 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a09064-5238-4ad0-bc87-ea6951d4c084 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:30.320945 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a09064-5238-4ad0-bc87-ea6951d4c084 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:30.328416 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:30.328885 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a09064-5238-4ad0-bc87-ea6951d4c084 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:30.330542 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a09064-5238-4ad0-bc87-ea6951d4c084 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:30.343646 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application [None req-f1a09064-5238-4ad0-bc87-ea6951d4c084 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 31 01:04:30.343646 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:04:30.343646 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:04:30.343646 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:04:30.343646 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:04:30.343646 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:04:30.343646 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:04:30.343646 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:04:30.343646 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:04:30.343646 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:04:30.343646 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:04:30.343646 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:04:30.343646 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 01:04:30.343646 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 01:04:30.343646 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 01:04:30.343646 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 01:04:30.344638 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:04:30.344638 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:04:30.344638 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 01:04:30.344638 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 01:04:30.344638 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 01:04:30.344638 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 01:04:30.344638 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:04:30.344638 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:04:30.344638 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:04:30.344638 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:04:30.344638 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:04:30.344638 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application with Lock( Aug 31 01:04:30.344638 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:04:30.344638 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:04:30.344638 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:04:30.344638 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:04:30.344638 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:04:30.344638 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:04:30.345973 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:04:30.345973 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:04:30.345973 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 01:04:30.345973 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 01:04:30.345973 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 01:04:30.345973 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 01:04:30.345973 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 01:04:30.345973 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 01:04:30.345973 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 31 01:04:30.345973 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Aug 31 01:04:30.348283 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 324/626] 10.222.0.20 () {56 vars in 1093 bytes} [Wed Aug 31 01:04:30 2022] GET /identity/v3/users/nova => generated 82 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:04:30.359022 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a09064-5238-4ad0-bc87-ea6951d4c084 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:30.366701 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6216ed94-d5e7-4451-ae85-d29932f4259f None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:30.432653 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68908568-f348-4dd0-935b-3912509c560e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:30.433785 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68908568-f348-4dd0-935b-3912509c560e None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:30.434071 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68908568-f348-4dd0-935b-3912509c560e None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:30.434408 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68908568-f348-4dd0-935b-3912509c560e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:30.443845 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68908568-f348-4dd0-935b-3912509c560e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:30.446684 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68908568-f348-4dd0-935b-3912509c560e None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:30.462796 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application [None req-68908568-f348-4dd0-935b-3912509c560e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 01:04:30.462796 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:04:30.462796 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:04:30.462796 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:04:30.462796 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:04:30.462796 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:04:30.462796 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:04:30.462796 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:04:30.462796 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:04:30.462796 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:04:30.462796 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:04:30.462796 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:04:30.462796 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 01:04:30.462796 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 01:04:30.462796 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 01:04:30.463712 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 01:04:30.463712 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:04:30.463712 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:04:30.463712 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:04:30.463712 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:04:30.463712 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:04:30.463712 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:04:30.463712 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:04:30.463712 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application with Lock( Aug 31 01:04:30.463712 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:04:30.463712 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:04:30.463712 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:04:30.463712 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:04:30.463712 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:04:30.463712 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:04:30.463712 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:04:30.463712 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:04:30.464627 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 01:04:30.464627 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 01:04:30.464627 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 01:04:30.464627 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 01:04:30.464627 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 01:04:30.464627 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 01:04:30.464627 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 01:04:30.464627 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Aug 31 01:04:30.464627 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 303/627] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Aug 31 01:04:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:04:30.465165 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054c0955-2f92-4682-995e-08ba52601320 None None] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:30.465686 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054c0955-2f92-4682-995e-08ba52601320 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:30.465879 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054c0955-2f92-4682-995e-08ba52601320 None None] PATH_INFO: `/` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:30.467390 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 304/628] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:04:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:04:30.470462 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054c0955-2f92-4682-995e-08ba52601320 None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:30.502837 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a09064-5238-4ad0-bc87-ea6951d4c084 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:30.527735 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:30.543826 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417b545a-3465-4bb5-a5aa-cc50c7016f37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:30.544441 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417b545a-3465-4bb5-a5aa-cc50c7016f37 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:30.544748 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417b545a-3465-4bb5-a5aa-cc50c7016f37 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:30.547370 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417b545a-3465-4bb5-a5aa-cc50c7016f37 None admin] PATH_INFO: `/v3/users` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:30.547370 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417b545a-3465-4bb5-a5aa-cc50c7016f37 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:04:30.547370 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417b545a-3465-4bb5-a5aa-cc50c7016f37 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:30.548395 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417b545a-3465-4bb5-a5aa-cc50c7016f37 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:30.573100 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 325/629] 10.222.0.20 () {56 vars in 1133 bytes} [Wed Aug 31 01:04:30 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 01:04:30.574898 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b1876a-ebc2-45ce-8c8f-7bff3389ce5a None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:30.575182 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b1876a-ebc2-45ce-8c8f-7bff3389ce5a None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:30.575454 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b1876a-ebc2-45ce-8c8f-7bff3389ce5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:30.596395 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-d5b1876a-ebc2-45ce-8c8f-7bff3389ce5a None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:04:30.597602 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-d5b1876a-ebc2-45ce-8c8f-7bff3389ce5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:04:30.605268 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054c0955-2f92-4682-995e-08ba52601320 None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:30.641044 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a28e45a-5c22-4ab6-bd9d-8bf60b7a684c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:30.641769 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a28e45a-5c22-4ab6-bd9d-8bf60b7a684c None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:30.642074 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a28e45a-5c22-4ab6-bd9d-8bf60b7a684c None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:30.642350 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a28e45a-5c22-4ab6-bd9d-8bf60b7a684c None admin] PATH_INFO: `/v3/projects` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:30.643038 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a28e45a-5c22-4ab6-bd9d-8bf60b7a684c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:04:30.643461 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a28e45a-5c22-4ab6-bd9d-8bf60b7a684c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:30.644917 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a28e45a-5c22-4ab6-bd9d-8bf60b7a684c None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:30.662788 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 305/630] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Aug 31 01:04:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:04:30.662788 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a28e45a-5c22-4ab6-bd9d-8bf60b7a684c None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:30.671740 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:30.685847 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 326/631] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:04:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 01:04:30.685847 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b1876a-ebc2-45ce-8c8f-7bff3389ce5a None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:30.774895 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a28e45a-5c22-4ab6-bd9d-8bf60b7a684c None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:30.816595 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b1876a-ebc2-45ce-8c8f-7bff3389ce5a None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:30.825905 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57a64c2-0761-4cef-a990-8a2f71d89a68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:30.826541 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57a64c2-0761-4cef-a990-8a2f71d89a68 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:30.826821 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57a64c2-0761-4cef-a990-8a2f71d89a68 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:30.827096 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57a64c2-0761-4cef-a990-8a2f71d89a68 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:30.832277 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:30.862031 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094f9817-6530-4dbf-8ce7-ae302d131195 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:30.863093 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094f9817-6530-4dbf-8ce7-ae302d131195 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:30.863587 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094f9817-6530-4dbf-8ce7-ae302d131195 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:30.864074 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094f9817-6530-4dbf-8ce7-ae302d131195 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:30.875434 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094f9817-6530-4dbf-8ce7-ae302d131195 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 01:04:30.881092 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application [None req-094f9817-6530-4dbf-8ce7-ae302d131195 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 01:04:30.881092 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:04:30.881092 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:04:30.881092 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:04:30.881092 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:04:30.881092 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:04:30.881092 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:04:30.881092 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:04:30.881092 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:04:30.881092 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:04:30.881092 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:04:30.881092 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:04:30.881092 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 01:04:30.881092 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 01:04:30.881092 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 01:04:30.881092 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 01:04:30.888575 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 01:04:30.888575 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 01:04:30.888575 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 01:04:30.888575 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 01:04:30.888575 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:04:30.888575 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:04:30.888575 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:04:30.888575 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:04:30.888575 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:04:30.888575 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:04:30.888575 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:04:30.888575 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application with Lock( Aug 31 01:04:30.888575 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:04:30.888575 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:04:30.888575 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:04:30.888575 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:04:30.889850 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:04:30.889850 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:04:30.889850 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:04:30.889850 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:04:30.889850 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 01:04:30.889850 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 01:04:30.889850 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 01:04:30.889850 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 01:04:30.889850 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 01:04:30.889850 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 01:04:30.889850 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 01:04:30.889850 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Aug 31 01:04:30.891117 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 327/632] 10.222.0.20 () {56 vars in 1102 bytes} [Wed Aug 31 01:04:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:04:30.892133 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094f9817-6530-4dbf-8ce7-ae302d131195 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:30.902020 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57a64c2-0761-4cef-a990-8a2f71d89a68 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:30.903543 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57a64c2-0761-4cef-a990-8a2f71d89a68 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:30.914063 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:30.997093 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094f9817-6530-4dbf-8ce7-ae302d131195 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:31.003953 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 306/633] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Aug 31 01:04:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 01:04:31.004628 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57a64c2-0761-4cef-a990-8a2f71d89a68 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:31.032882 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eab4632-12f9-4c9d-ad36-588d4ccba4c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:31.033540 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eab4632-12f9-4c9d-ad36-588d4ccba4c0 None admin] REQUEST_METHOD: `PUT` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:31.033827 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eab4632-12f9-4c9d-ad36-588d4ccba4c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:31.034105 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eab4632-12f9-4c9d-ad36-588d4ccba4c0 None admin] PATH_INFO: `/v3/projects/60b2abc15c2849ba8a1ed92117cc981b/users/fc1d010919ec40878ae829cc8dceec4c/roles/7d00a4cfa13744a69604ac2cf1a5816e` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:31.050987 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eab4632-12f9-4c9d-ad36-588d4ccba4c0 None admin] RBAC: Authorizing `identity:create_grant(project_id=60b2abc15c2849ba8a1ed92117cc981b, user_id=fc1d010919ec40878ae829cc8dceec4c, role_id=7d00a4cfa13744a69604ac2cf1a5816e)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:31.052418 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eab4632-12f9-4c9d-ad36-588d4ccba4c0 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:31.102786 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57a64c2-0761-4cef-a990-8a2f71d89a68 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:31.111510 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:31.135487 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0861bc10-6b76-4cd2-a993-025a9e6cfe4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:31.136083 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0861bc10-6b76-4cd2-a993-025a9e6cfe4e None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:31.136364 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0861bc10-6b76-4cd2-a993-025a9e6cfe4e None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:31.136638 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0861bc10-6b76-4cd2-a993-025a9e6cfe4e None admin] PATH_INFO: `/v3/roles` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:31.137322 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0861bc10-6b76-4cd2-a993-025a9e6cfe4e None admin] RBAC: Adding query filter params (name=service) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:04:31.137626 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0861bc10-6b76-4cd2-a993-025a9e6cfe4e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:31.138998 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0861bc10-6b76-4cd2-a993-025a9e6cfe4e None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:31.146539 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 307/634] 10.222.0.20 () {56 vars in 1103 bytes} [Wed Aug 31 01:04:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:04:31.147786 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37a9354-68ba-4d1f-818b-fda57dd8cff3 None None] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:31.148145 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37a9354-68ba-4d1f-818b-fda57dd8cff3 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:31.148422 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37a9354-68ba-4d1f-818b-fda57dd8cff3 None None] PATH_INFO: `/` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:31.149637 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 308/635] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:04:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:04:31.150331 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37a9354-68ba-4d1f-818b-fda57dd8cff3 None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:31.201147 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 328/636] 10.222.0.20 () {58 vars in 1439 bytes} [Wed Aug 31 01:04:30 2022] PUT /identity/v3/projects/60b2abc15c2849ba8a1ed92117cc981b/users/fc1d010919ec40878ae829cc8dceec4c/roles/7d00a4cfa13744a69604ac2cf1a5816e => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:04:31.201984 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eab4632-12f9-4c9d-ad36-588d4ccba4c0 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:31.226922 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:31.265425 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37a9354-68ba-4d1f-818b-fda57dd8cff3 None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:31.302267 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ea301e-5039-44a2-a1e6-112a4734753b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:31.302919 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ea301e-5039-44a2-a1e6-112a4734753b None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:31.303203 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ea301e-5039-44a2-a1e6-112a4734753b None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:31.303481 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ea301e-5039-44a2-a1e6-112a4734753b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:31.315510 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ea301e-5039-44a2-a1e6-112a4734753b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 01:04:31.320854 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eab4632-12f9-4c9d-ad36-588d4ccba4c0 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:31.321063 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application [None req-50ea301e-5039-44a2-a1e6-112a4734753b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 01:04:31.321063 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:04:31.321063 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:04:31.321063 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:04:31.321063 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:04:31.321063 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:04:31.321063 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:04:31.321063 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:04:31.321063 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:04:31.321063 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:04:31.321063 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:04:31.321063 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:04:31.321063 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 01:04:31.321063 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 01:04:31.321063 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 01:04:31.321063 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 01:04:31.322105 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 01:04:31.322105 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 01:04:31.322105 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 01:04:31.322105 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 01:04:31.322105 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:04:31.322105 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:04:31.322105 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:04:31.322105 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:04:31.322105 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:04:31.322105 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:04:31.322105 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:04:31.322105 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application with Lock( Aug 31 01:04:31.322105 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:04:31.322105 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:04:31.322105 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:04:31.322105 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:04:31.323088 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:04:31.323088 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:04:31.323088 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:04:31.323088 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:04:31.323088 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 01:04:31.323088 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 01:04:31.323088 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 01:04:31.323088 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 01:04:31.323088 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 01:04:31.323088 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 01:04:31.323088 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 01:04:31.323088 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Aug 31 01:04:31.323088 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 309/637] 10.222.0.20 () {56 vars in 1102 bytes} [Wed Aug 31 01:04:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:04:31.323088 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ea301e-5039-44a2-a1e6-112a4734753b None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:31.366578 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:31.370021 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce81324-8c06-4894-9c24-bb7f25da54ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:31.370748 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce81324-8c06-4894-9c24-bb7f25da54ed None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:31.371032 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce81324-8c06-4894-9c24-bb7f25da54ed None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:31.371306 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce81324-8c06-4894-9c24-bb7f25da54ed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:31.382130 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce81324-8c06-4894-9c24-bb7f25da54ed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:31.428068 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce81324-8c06-4894-9c24-bb7f25da54ed None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:31.428068 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application [None req-5ce81324-8c06-4894-9c24-bb7f25da54ed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 01:04:31.428068 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:04:31.428068 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:04:31.428068 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:04:31.428068 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:04:31.428068 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:04:31.428068 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:04:31.428068 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:04:31.428068 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:04:31.428068 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:04:31.428068 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:04:31.428068 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:04:31.428068 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 01:04:31.429173 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 01:04:31.429173 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 01:04:31.429173 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 01:04:31.429173 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:04:31.429173 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:04:31.429173 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:04:31.429173 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:04:31.429173 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:04:31.429173 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:04:31.429173 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:04:31.429173 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application with Lock( Aug 31 01:04:31.429173 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:04:31.429173 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:04:31.429173 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:04:31.429173 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:04:31.429173 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:04:31.429173 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:04:31.430615 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:04:31.430615 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:04:31.430615 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 01:04:31.430615 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 01:04:31.430615 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 01:04:31.430615 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Aug 31 01:04:31.430615 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 329/638] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 01:04:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:04:31.430615 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0419d964-cd5c-4bbd-aba1-a794d58cf1c4 None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:31.430615 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0419d964-cd5c-4bbd-aba1-a794d58cf1c4 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:31.430615 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0419d964-cd5c-4bbd-aba1-a794d58cf1c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:31.431202 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-0419d964-cd5c-4bbd-aba1-a794d58cf1c4 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:04:31.431202 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-0419d964-cd5c-4bbd-aba1-a794d58cf1c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:04:31.446130 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ea301e-5039-44a2-a1e6-112a4734753b None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:31.484357 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f7e902-4401-4d02-b663-036fb2a2ffaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:31.485134 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f7e902-4401-4d02-b663-036fb2a2ffaa None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:31.485416 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f7e902-4401-4d02-b663-036fb2a2ffaa None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:31.485811 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f7e902-4401-4d02-b663-036fb2a2ffaa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:31.492854 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 330/639] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:04:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 01:04:31.493967 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0419d964-cd5c-4bbd-aba1-a794d58cf1c4 None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:31.496312 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:31.501418 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f7e902-4401-4d02-b663-036fb2a2ffaa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:31.502881 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f7e902-4401-4d02-b663-036fb2a2ffaa None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:31.518388 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application [None req-74f7e902-4401-4d02-b663-036fb2a2ffaa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 01:04:31.518388 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:04:31.518388 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:04:31.518388 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:04:31.518388 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:04:31.518388 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:04:31.518388 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:04:31.518388 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:04:31.518388 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:04:31.518388 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:04:31.518388 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:04:31.518388 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:04:31.518388 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 01:04:31.518388 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 01:04:31.518388 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 01:04:31.518388 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 01:04:31.519347 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:04:31.519347 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:04:31.519347 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:04:31.519347 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:04:31.519347 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:04:31.519347 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:04:31.519347 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:04:31.519347 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application with Lock( Aug 31 01:04:31.519347 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:04:31.519347 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:04:31.519347 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:04:31.519347 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:04:31.519347 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:04:31.519347 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:04:31.519347 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:04:31.519347 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:04:31.519347 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 01:04:31.520263 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 01:04:31.520263 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 01:04:31.520263 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Aug 31 01:04:31.524771 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 310/640] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 01:04:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:04:31.525805 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f7e902-4401-4d02-b663-036fb2a2ffaa None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:31.615615 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:31.629801 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0419d964-cd5c-4bbd-aba1-a794d58cf1c4 None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:31.635678 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f7e902-4401-4d02-b663-036fb2a2ffaa None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:31.671388 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ff5f29-d06a-473b-93c8-7b1161709cfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:31.672101 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ff5f29-d06a-473b-93c8-7b1161709cfe None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:31.672454 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ff5f29-d06a-473b-93c8-7b1161709cfe None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:31.672998 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ff5f29-d06a-473b-93c8-7b1161709cfe None admin] PATH_INFO: `/v3/roles` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:31.673570 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ff5f29-d06a-473b-93c8-7b1161709cfe None admin] RBAC: Adding query filter params (name=service) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:04:31.673990 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ff5f29-d06a-473b-93c8-7b1161709cfe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:31.675524 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ff5f29-d06a-473b-93c8-7b1161709cfe None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:31.678947 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4a0531-8b95-4118-b684-50f76f9a98f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:31.680631 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4a0531-8b95-4118-b684-50f76f9a98f3 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:31.681131 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4a0531-8b95-4118-b684-50f76f9a98f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:31.681597 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4a0531-8b95-4118-b684-50f76f9a98f3 None admin] PATH_INFO: `/v3/domains` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:31.682857 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4a0531-8b95-4118-b684-50f76f9a98f3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:04:31.683605 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4a0531-8b95-4118-b684-50f76f9a98f3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:31.685189 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4a0531-8b95-4118-b684-50f76f9a98f3 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:31.685564 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 331/641] 10.222.0.20 () {56 vars in 1103 bytes} [Wed Aug 31 01:04:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:04:31.686527 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ff5f29-d06a-473b-93c8-7b1161709cfe None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:31.702496 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 311/642] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 31 01:04:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:04:31.704085 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4a0531-8b95-4118-b684-50f76f9a98f3 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:31.763749 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:31.813393 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ff5f29-d06a-473b-93c8-7b1161709cfe None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:31.827213 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4a0531-8b95-4118-b684-50f76f9a98f3 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:31.849074 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e5ccd5-c714-41cf-9778-4876ba947b61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:31.851152 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e5ccd5-c714-41cf-9778-4876ba947b61 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:31.851152 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e5ccd5-c714-41cf-9778-4876ba947b61 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:31.851152 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e5ccd5-c714-41cf-9778-4876ba947b61 None admin] PATH_INFO: `/v3/domains` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:31.851152 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e5ccd5-c714-41cf-9778-4876ba947b61 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:04:31.852013 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e5ccd5-c714-41cf-9778-4876ba947b61 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:31.853121 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e5ccd5-c714-41cf-9778-4876ba947b61 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:31.869349 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 332/643] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 31 01:04:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:04:31.869349 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e5ccd5-c714-41cf-9778-4876ba947b61 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:31.872350 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90744dc1-3110-4b4b-a702-6fd7a12fbe1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:31.873014 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90744dc1-3110-4b4b-a702-6fd7a12fbe1a None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:31.873298 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90744dc1-3110-4b4b-a702-6fd7a12fbe1a None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:31.873629 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90744dc1-3110-4b4b-a702-6fd7a12fbe1a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:31.884832 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90744dc1-3110-4b4b-a702-6fd7a12fbe1a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:31.886319 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90744dc1-3110-4b4b-a702-6fd7a12fbe1a None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:31.888277 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:31.903374 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application [None req-90744dc1-3110-4b4b-a702-6fd7a12fbe1a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 01:04:31.903374 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:04:31.903374 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:04:31.903374 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:04:31.903374 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:04:31.903374 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:04:31.903374 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:04:31.903374 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:04:31.903374 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:04:31.903374 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:04:31.903374 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:04:31.903374 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:04:31.903374 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 01:04:31.903374 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 01:04:31.903374 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 01:04:31.903374 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 01:04:31.910068 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:04:31.910068 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:04:31.910068 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:04:31.910068 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:04:31.910068 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:04:31.910068 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:04:31.910068 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:04:31.910068 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application with Lock( Aug 31 01:04:31.910068 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:04:31.910068 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:04:31.910068 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:04:31.910068 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:04:31.910068 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:04:31.910068 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:04:31.910068 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:04:31.910068 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:04:31.910068 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 01:04:31.910979 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 01:04:31.910979 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 01:04:31.910979 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Aug 31 01:04:31.910979 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 312/644] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 01:04:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:04:31.910979 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90744dc1-3110-4b4b-a702-6fd7a12fbe1a None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:31.985691 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e5ccd5-c714-41cf-9778-4876ba947b61 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:32.007260 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90744dc1-3110-4b4b-a702-6fd7a12fbe1a None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:32.009059 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:32.020698 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09be71f-b934-4e57-973d-121cc4153aa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:32.022802 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09be71f-b934-4e57-973d-121cc4153aa7 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:32.022802 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09be71f-b934-4e57-973d-121cc4153aa7 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:32.022802 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09be71f-b934-4e57-973d-121cc4153aa7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:32.029484 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09be71f-b934-4e57-973d-121cc4153aa7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:32.030930 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09be71f-b934-4e57-973d-121cc4153aa7 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:32.043051 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application [None req-f09be71f-b934-4e57-973d-121cc4153aa7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 01:04:32.043051 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:04:32.043051 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:04:32.043051 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:04:32.043051 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:04:32.043051 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:04:32.043051 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:04:32.043051 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:04:32.043051 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:04:32.043051 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:04:32.043051 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:04:32.043051 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:04:32.043051 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 01:04:32.043051 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 01:04:32.043051 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 01:04:32.044000 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 01:04:32.044000 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:04:32.044000 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:04:32.044000 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:04:32.044000 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:04:32.044000 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:04:32.044000 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:04:32.044000 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:04:32.044000 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application with Lock( Aug 31 01:04:32.044000 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:04:32.044000 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:04:32.044000 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:04:32.044000 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:04:32.044000 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:04:32.044000 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:04:32.044000 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:04:32.044000 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:04:32.045549 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 01:04:32.045549 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 01:04:32.045549 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 01:04:32.045549 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 01:04:32.045549 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 01:04:32.045549 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 01:04:32.045549 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 01:04:32.045549 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Aug 31 01:04:32.046798 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bf99ef-2c14-44fa-8756-ed7be71900ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:32.047433 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bf99ef-2c14-44fa-8756-ed7be71900ca None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:32.047714 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bf99ef-2c14-44fa-8756-ed7be71900ca None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:32.047994 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bf99ef-2c14-44fa-8756-ed7be71900ca None admin] PATH_INFO: `/v3/roles/service` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:32.048848 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 333/645] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Aug 31 01:04:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:04:32.049549 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09be71f-b934-4e57-973d-121cc4153aa7 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:32.059504 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bf99ef-2c14-44fa-8756-ed7be71900ca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 01:04:32.065387 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application [None req-d5bf99ef-2c14-44fa-8756-ed7be71900ca None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 01:04:32.065387 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:04:32.065387 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:04:32.065387 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:04:32.065387 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:04:32.065387 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:04:32.065387 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:04:32.065387 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:04:32.065387 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:04:32.065387 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:04:32.065387 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:04:32.065387 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:04:32.065387 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 01:04:32.065387 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 01:04:32.065387 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 01:04:32.065387 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 01:04:32.066318 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 01:04:32.066318 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 01:04:32.066318 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 01:04:32.066318 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 01:04:32.066318 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:04:32.066318 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:04:32.066318 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:04:32.066318 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:04:32.066318 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:04:32.066318 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:04:32.066318 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:04:32.066318 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application with Lock( Aug 31 01:04:32.066318 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:04:32.066318 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:04:32.066318 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:04:32.066318 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:04:32.067750 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:04:32.067750 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:04:32.067750 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:04:32.067750 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:04:32.067750 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 01:04:32.067750 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 01:04:32.067750 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 01:04:32.067750 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 01:04:32.067750 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 01:04:32.067750 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 01:04:32.067750 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 01:04:32.067750 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Aug 31 01:04:32.067750 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 313/646] 10.222.0.20 () {56 vars in 1102 bytes} [Wed Aug 31 01:04:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:04:32.067750 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bf99ef-2c14-44fa-8756-ed7be71900ca None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:32.126586 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:32.153849 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09be71f-b934-4e57-973d-121cc4153aa7 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:32.173801 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bf99ef-2c14-44fa-8756-ed7be71900ca None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:32.188016 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f3132b-d64a-4b26-9131-a958371074da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:32.188650 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f3132b-d64a-4b26-9131-a958371074da None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:32.188958 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f3132b-d64a-4b26-9131-a958371074da None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:32.189234 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f3132b-d64a-4b26-9131-a958371074da None admin] PATH_INFO: `/v3/users/glance` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:32.198313 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f3132b-d64a-4b26-9131-a958371074da None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:32.200018 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f3132b-d64a-4b26-9131-a958371074da None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:32.207652 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33ba1e9-3312-4e8d-ac46-af5778a73ead None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:32.208252 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33ba1e9-3312-4e8d-ac46-af5778a73ead None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:32.208531 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33ba1e9-3312-4e8d-ac46-af5778a73ead None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:32.208831 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33ba1e9-3312-4e8d-ac46-af5778a73ead None admin] PATH_INFO: `/v3/domains` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:32.209470 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33ba1e9-3312-4e8d-ac46-af5778a73ead None admin] RBAC: Adding query filter params (name=Default) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:04:32.209772 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33ba1e9-3312-4e8d-ac46-af5778a73ead None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:32.211683 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33ba1e9-3312-4e8d-ac46-af5778a73ead None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:32.211852 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application [None req-c9f3132b-d64a-4b26-9131-a958371074da None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 31 01:04:32.211852 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:04:32.211852 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:04:32.211852 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:04:32.211852 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:04:32.211852 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:04:32.211852 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:04:32.211852 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:04:32.211852 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:04:32.211852 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:04:32.211852 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:04:32.211852 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:04:32.211852 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 01:04:32.211852 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 01:04:32.211852 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 01:04:32.211852 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 01:04:32.212822 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:04:32.212822 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:04:32.212822 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 01:04:32.212822 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 01:04:32.212822 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 01:04:32.212822 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 01:04:32.212822 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:04:32.212822 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:04:32.212822 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:04:32.212822 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:04:32.212822 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:04:32.212822 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application with Lock( Aug 31 01:04:32.212822 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:04:32.212822 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:04:32.212822 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:04:32.212822 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:04:32.212822 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:04:32.212822 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:04:32.213795 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:04:32.213795 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:04:32.213795 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 01:04:32.213795 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 01:04:32.213795 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 01:04:32.213795 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 01:04:32.213795 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 01:04:32.213795 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 01:04:32.213795 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 31 01:04:32.213795 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Aug 31 01:04:32.213795 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 334/647] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Aug 31 01:04:32 2022] GET /identity/v3/users/glance => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:04:32.213795 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f3132b-d64a-4b26-9131-a958371074da None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:32.233045 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 314/648] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 31 01:04:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:04:32.233045 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33ba1e9-3312-4e8d-ac46-af5778a73ead None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:32.266373 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:32.328025 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f3132b-d64a-4b26-9131-a958371074da None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:32.334001 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33ba1e9-3312-4e8d-ac46-af5778a73ead None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:32.349064 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:32.367929 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650eb656-cb4d-4018-9092-41f0bf79fba6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:32.369133 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d405c1-22e4-4412-860b-5db63a7b0821 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:32.369607 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650eb656-cb4d-4018-9092-41f0bf79fba6 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:32.369782 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d405c1-22e4-4412-860b-5db63a7b0821 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:32.370023 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d405c1-22e4-4412-860b-5db63a7b0821 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:32.370299 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d405c1-22e4-4412-860b-5db63a7b0821 None admin] PATH_INFO: `/v3/roles` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:32.370639 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650eb656-cb4d-4018-9092-41f0bf79fba6 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:32.370989 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d405c1-22e4-4412-860b-5db63a7b0821 None admin] RBAC: Adding query filter params (name=service) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:04:32.371301 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d405c1-22e4-4412-860b-5db63a7b0821 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:32.371614 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650eb656-cb4d-4018-9092-41f0bf79fba6 None admin] PATH_INFO: `/v3/projects` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:32.372483 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650eb656-cb4d-4018-9092-41f0bf79fba6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:04:32.372741 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d405c1-22e4-4412-860b-5db63a7b0821 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:32.373219 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650eb656-cb4d-4018-9092-41f0bf79fba6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:32.375348 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650eb656-cb4d-4018-9092-41f0bf79fba6 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:32.380844 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 315/649] 10.222.0.20 () {56 vars in 1103 bytes} [Wed Aug 31 01:04:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:04:32.381636 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d405c1-22e4-4412-860b-5db63a7b0821 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:32.392604 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 335/650] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Aug 31 01:04:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:04:32.393811 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650eb656-cb4d-4018-9092-41f0bf79fba6 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:32.495861 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d405c1-22e4-4412-860b-5db63a7b0821 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:32.499175 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650eb656-cb4d-4018-9092-41f0bf79fba6 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:32.501462 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:32.533584 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c435c2-cde7-4db8-81c0-9902816a7528 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:32.534848 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c435c2-cde7-4db8-81c0-9902816a7528 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:32.535135 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c435c2-cde7-4db8-81c0-9902816a7528 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:32.535459 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c435c2-cde7-4db8-81c0-9902816a7528 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:32.536581 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e1622c-b55c-4490-8842-7192f2d85c73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:32.537378 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e1622c-b55c-4490-8842-7192f2d85c73 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:32.537665 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e1622c-b55c-4490-8842-7192f2d85c73 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:32.538421 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e1622c-b55c-4490-8842-7192f2d85c73 None admin] PATH_INFO: `/v3/users` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:32.539223 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e1622c-b55c-4490-8842-7192f2d85c73 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:04:32.539535 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e1622c-b55c-4490-8842-7192f2d85c73 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:32.540999 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e1622c-b55c-4490-8842-7192f2d85c73 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:32.542565 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c435c2-cde7-4db8-81c0-9902816a7528 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:32.544121 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c435c2-cde7-4db8-81c0-9902816a7528 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:32.556135 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application [None req-13c435c2-cde7-4db8-81c0-9902816a7528 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 31 01:04:32.556135 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:04:32.556135 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:04:32.556135 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:04:32.556135 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:04:32.556135 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:04:32.556135 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:04:32.556135 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:04:32.556135 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:04:32.556135 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:04:32.556135 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:04:32.556135 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:04:32.556135 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 01:04:32.556135 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 01:04:32.556135 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 01:04:32.556135 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 01:04:32.557266 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:04:32.557266 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:04:32.557266 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 01:04:32.557266 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 01:04:32.557266 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 01:04:32.557266 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 01:04:32.557266 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:04:32.557266 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:04:32.557266 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:04:32.557266 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:04:32.557266 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:04:32.557266 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application with Lock( Aug 31 01:04:32.557266 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:04:32.557266 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:04:32.557266 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:04:32.557266 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:04:32.557266 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:04:32.557266 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:04:32.558487 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:04:32.558487 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:04:32.558487 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 01:04:32.558487 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 01:04:32.558487 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 01:04:32.558487 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 01:04:32.558487 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 01:04:32.558487 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 01:04:32.558487 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 31 01:04:32.558487 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Aug 31 01:04:32.560865 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 336/651] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Aug 31 01:04:32 2022] GET /identity/v3/users/cinder => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:04:32.566022 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 316/652] 10.222.0.20 () {56 vars in 1137 bytes} [Wed Aug 31 01:04:32 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:04:32.566022 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e1622c-b55c-4490-8842-7192f2d85c73 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:32.566237 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c435c2-cde7-4db8-81c0-9902816a7528 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:32.600291 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:32.667575 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c435c2-cde7-4db8-81c0-9902816a7528 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:32.671294 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e1622c-b55c-4490-8842-7192f2d85c73 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:32.701726 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008172ac-059a-4d6c-b24a-a46c1cf3c99b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:32.702364 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008172ac-059a-4d6c-b24a-a46c1cf3c99b None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:32.702649 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008172ac-059a-4d6c-b24a-a46c1cf3c99b None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:32.702925 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008172ac-059a-4d6c-b24a-a46c1cf3c99b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:32.707594 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8b57db-ed5e-4532-8348-f49878ad1a68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:32.708246 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8b57db-ed5e-4532-8348-f49878ad1a68 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:32.708486 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8b57db-ed5e-4532-8348-f49878ad1a68 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:32.708784 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8b57db-ed5e-4532-8348-f49878ad1a68 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:32.709518 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8b57db-ed5e-4532-8348-f49878ad1a68 None admin] RBAC: Adding query filter params (role.id=7d00a4cfa13744a69604ac2cf1a5816e, scope.project.id=60b2abc15c2849ba8a1ed92117cc981b, user.id=8744c9de803048f583d0317d01f6a540) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:04:32.709831 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8b57db-ed5e-4532-8348-f49878ad1a68 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:32.711303 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8b57db-ed5e-4532-8348-f49878ad1a68 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:32.713803 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008172ac-059a-4d6c-b24a-a46c1cf3c99b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:32.715253 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008172ac-059a-4d6c-b24a-a46c1cf3c99b None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:32.717493 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 317/653] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Aug 31 01:04:32 2022] GET /identity/v3/role_assignments?user.id=8744c9de803048f583d0317d01f6a540&scope.project.id=60b2abc15c2849ba8a1ed92117cc981b&role.id=7d00a4cfa13744a69604ac2cf1a5816e => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:04:32.718466 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8b57db-ed5e-4532-8348-f49878ad1a68 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:32.743163 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application [None req-008172ac-059a-4d6c-b24a-a46c1cf3c99b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 01:04:32.743163 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:04:32.743163 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:04:32.743163 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:04:32.743163 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:04:32.743163 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:04:32.743163 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:04:32.743163 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:04:32.743163 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:04:32.743163 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:04:32.743163 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:04:32.743163 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:04:32.743163 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 01:04:32.743163 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 01:04:32.743163 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 01:04:32.743163 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 01:04:32.744394 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:04:32.744394 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:04:32.744394 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:04:32.744394 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:04:32.744394 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:04:32.744394 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:04:32.744394 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:04:32.744394 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application with Lock( Aug 31 01:04:32.744394 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:04:32.744394 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:04:32.744394 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:04:32.744394 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:04:32.744394 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:04:32.744394 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:04:32.744394 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:04:32.744394 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:04:32.744394 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 01:04:32.745558 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 01:04:32.745558 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 01:04:32.745558 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Aug 31 01:04:32.752032 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 337/654] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 01:04:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:04:32.753270 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008172ac-059a-4d6c-b24a-a46c1cf3c99b None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:32.772688 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:32.853937 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:32.859565 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8b57db-ed5e-4532-8348-f49878ad1a68 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:32.887400 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008172ac-059a-4d6c-b24a-a46c1cf3c99b None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:32.916913 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2062a029-50f8-4444-b070-71259b80f481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:32.917673 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2062a029-50f8-4444-b070-71259b80f481 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:32.918061 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2062a029-50f8-4444-b070-71259b80f481 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:32.918440 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2062a029-50f8-4444-b070-71259b80f481 None admin] PATH_INFO: `/v3/users` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:32.919268 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2062a029-50f8-4444-b070-71259b80f481 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:04:32.919647 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2062a029-50f8-4444-b070-71259b80f481 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:32.921206 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2062a029-50f8-4444-b070-71259b80f481 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:32.933639 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0755ee5-c6b4-4364-a726-7cd413033c9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:32.934256 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0755ee5-c6b4-4364-a726-7cd413033c9a None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:32.934539 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0755ee5-c6b4-4364-a726-7cd413033c9a None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:32.934839 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0755ee5-c6b4-4364-a726-7cd413033c9a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:32.947210 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 318/655] 10.222.0.20 () {56 vars in 1137 bytes} [Wed Aug 31 01:04:32 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:04:32.947972 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2062a029-50f8-4444-b070-71259b80f481 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:33.023920 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0755ee5-c6b4-4364-a726-7cd413033c9a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:33.026036 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0755ee5-c6b4-4364-a726-7cd413033c9a None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:33.046013 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:33.082009 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2062a029-50f8-4444-b070-71259b80f481 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:33.133385 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 338/656] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Aug 31 01:04:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 01:04:33.133560 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01407722-70ca-46ab-a61f-28e72b5afab8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:33.134303 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01407722-70ca-46ab-a61f-28e72b5afab8 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:33.134655 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01407722-70ca-46ab-a61f-28e72b5afab8 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:33.135023 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01407722-70ca-46ab-a61f-28e72b5afab8 None admin] PATH_INFO: `/v3/domains` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:33.135265 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0755ee5-c6b4-4364-a726-7cd413033c9a None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:33.135863 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01407722-70ca-46ab-a61f-28e72b5afab8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:04:33.136199 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01407722-70ca-46ab-a61f-28e72b5afab8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:33.137692 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01407722-70ca-46ab-a61f-28e72b5afab8 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:33.152293 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 319/657] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 31 01:04:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:04:33.154309 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd9d889-4537-4cf7-9877-493464402760 None None] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:33.154716 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd9d889-4537-4cf7-9877-493464402760 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:33.155069 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd9d889-4537-4cf7-9877-493464402760 None None] PATH_INFO: `/` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:33.157057 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 320/658] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:04:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:04:33.157867 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd9d889-4537-4cf7-9877-493464402760 None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:33.178264 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:33.245223 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0755ee5-c6b4-4364-a726-7cd413033c9a None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:33.262604 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:33.267873 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd9d889-4537-4cf7-9877-493464402760 None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:33.280472 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804b162b-b7bd-438b-b1ae-b922ee4fa24e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:33.281122 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804b162b-b7bd-438b-b1ae-b922ee4fa24e None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:33.281409 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804b162b-b7bd-438b-b1ae-b922ee4fa24e None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:33.281686 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804b162b-b7bd-438b-b1ae-b922ee4fa24e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:33.306409 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dad9b0e-924d-4e0e-a87a-9e451e7a6f25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:33.307284 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dad9b0e-924d-4e0e-a87a-9e451e7a6f25 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:33.307792 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dad9b0e-924d-4e0e-a87a-9e451e7a6f25 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:33.308208 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dad9b0e-924d-4e0e-a87a-9e451e7a6f25 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:33.320019 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dad9b0e-924d-4e0e-a87a-9e451e7a6f25 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:33.321770 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dad9b0e-924d-4e0e-a87a-9e451e7a6f25 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:33.337680 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application [None req-9dad9b0e-924d-4e0e-a87a-9e451e7a6f25 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 01:04:33.337680 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:04:33.337680 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:04:33.337680 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:04:33.337680 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:04:33.337680 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:04:33.337680 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:04:33.337680 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:04:33.337680 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:04:33.337680 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:04:33.337680 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:04:33.337680 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:04:33.337680 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 01:04:33.337680 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 01:04:33.337680 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 01:04:33.337680 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 01:04:33.338838 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:04:33.338838 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:04:33.338838 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:04:33.338838 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:04:33.338838 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:04:33.338838 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:04:33.338838 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:04:33.338838 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application with Lock( Aug 31 01:04:33.338838 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:04:33.338838 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:04:33.338838 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:04:33.338838 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:04:33.338838 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:04:33.338838 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:04:33.338838 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:04:33.338838 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:04:33.338838 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 01:04:33.339742 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 01:04:33.339742 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 01:04:33.339742 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Aug 31 01:04:33.341124 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 321/659] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 01:04:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:04:33.342071 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dad9b0e-924d-4e0e-a87a-9e451e7a6f25 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:33.359901 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804b162b-b7bd-438b-b1ae-b922ee4fa24e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:33.361382 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804b162b-b7bd-438b-b1ae-b922ee4fa24e None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:33.413335 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:33.446888 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dad9b0e-924d-4e0e-a87a-9e451e7a6f25 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:33.467726 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 339/660] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Aug 31 01:04:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 01:04:33.469197 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275920dc-ad79-4237-bf38-5e924e2e8c47 None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:33.469487 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275920dc-ad79-4237-bf38-5e924e2e8c47 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:33.469760 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275920dc-ad79-4237-bf38-5e924e2e8c47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:33.485777 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcb3491-6529-4e93-9388-c797f8e603e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:33.486409 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcb3491-6529-4e93-9388-c797f8e603e5 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:33.486693 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcb3491-6529-4e93-9388-c797f8e603e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:33.486968 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcb3491-6529-4e93-9388-c797f8e603e5 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:33.487206 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-275920dc-ad79-4237-bf38-5e924e2e8c47 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:04:33.488373 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-275920dc-ad79-4237-bf38-5e924e2e8c47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:04:33.494657 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dcb3491-6529-4e93-9388-c797f8e603e5 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:33.496114 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dcb3491-6529-4e93-9388-c797f8e603e5 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:33.499848 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:33.508188 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application [None req-0dcb3491-6529-4e93-9388-c797f8e603e5 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 31 01:04:33.508188 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:04:33.508188 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:04:33.508188 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:04:33.508188 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:04:33.508188 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:04:33.508188 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:04:33.508188 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:04:33.508188 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:04:33.508188 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:04:33.508188 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:04:33.508188 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:04:33.508188 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 01:04:33.508188 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 01:04:33.508188 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 01:04:33.508188 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 01:04:33.515716 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:04:33.515716 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:04:33.515716 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 01:04:33.515716 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 01:04:33.515716 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 01:04:33.515716 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 01:04:33.515716 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:04:33.515716 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:04:33.515716 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:04:33.515716 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:04:33.515716 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:04:33.515716 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application with Lock( Aug 31 01:04:33.515716 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:04:33.515716 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:04:33.515716 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:04:33.515716 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:04:33.515716 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:04:33.515716 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:04:33.516713 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:04:33.516713 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:04:33.516713 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 01:04:33.516713 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 01:04:33.516713 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 01:04:33.516713 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 01:04:33.516713 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 01:04:33.516713 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 01:04:33.516713 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 31 01:04:33.516713 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Aug 31 01:04:33.516713 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 322/661] 10.222.0.20 () {56 vars in 1102 bytes} [Wed Aug 31 01:04:33 2022] GET /identity/v3/users/neutron => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:04:33.516713 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcb3491-6529-4e93-9388-c797f8e603e5 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:33.562266 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 340/662] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:04:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 01:04:33.563464 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275920dc-ad79-4237-bf38-5e924e2e8c47 None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:33.626013 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcb3491-6529-4e93-9388-c797f8e603e5 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:33.655573 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:33.666303 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9c13b7-03d8-444c-8bd6-d6a9029387d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:33.666303 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9c13b7-03d8-444c-8bd6-d6a9029387d4 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:33.666303 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9c13b7-03d8-444c-8bd6-d6a9029387d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:33.666303 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9c13b7-03d8-444c-8bd6-d6a9029387d4 None admin] PATH_INFO: `/v3/domains` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:33.666662 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9c13b7-03d8-444c-8bd6-d6a9029387d4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:04:33.666662 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9c13b7-03d8-444c-8bd6-d6a9029387d4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:33.667820 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9c13b7-03d8-444c-8bd6-d6a9029387d4 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:33.680208 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 323/663] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 31 01:04:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:04:33.681353 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9c13b7-03d8-444c-8bd6-d6a9029387d4 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:33.703931 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275920dc-ad79-4237-bf38-5e924e2e8c47 None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:33.753183 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dbfbbc-51b1-40e2-8324-d334de5c9dce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:33.753861 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dbfbbc-51b1-40e2-8324-d334de5c9dce None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:33.754145 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dbfbbc-51b1-40e2-8324-d334de5c9dce None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:33.754421 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dbfbbc-51b1-40e2-8324-d334de5c9dce None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:33.769077 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06dbfbbc-51b1-40e2-8324-d334de5c9dce None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:33.770941 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06dbfbbc-51b1-40e2-8324-d334de5c9dce None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:33.780712 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:33.791228 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9c13b7-03d8-444c-8bd6-d6a9029387d4 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:33.792406 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application [None req-06dbfbbc-51b1-40e2-8324-d334de5c9dce None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 01:04:33.792406 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:04:33.792406 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:04:33.792406 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:04:33.792406 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:04:33.792406 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:04:33.792406 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:04:33.792406 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:04:33.792406 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:04:33.792406 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:04:33.792406 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:04:33.792406 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:04:33.792406 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 01:04:33.792406 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 01:04:33.792406 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 01:04:33.792406 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 01:04:33.793362 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:04:33.793362 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:04:33.793362 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:04:33.793362 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:04:33.793362 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:04:33.793362 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:04:33.793362 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:04:33.793362 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application with Lock( Aug 31 01:04:33.793362 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:04:33.793362 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:04:33.793362 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:04:33.793362 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:04:33.793362 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:04:33.793362 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:04:33.793362 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:04:33.793362 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:04:33.793362 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 01:04:33.794805 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 01:04:33.794805 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 01:04:33.794805 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Aug 31 01:04:33.794805 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 341/664] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 01:04:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:04:33.794805 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dbfbbc-51b1-40e2-8324-d334de5c9dce None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:33.833886 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9379bd2f-e4d3-42e1-ac64-1f74de859cf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:33.834882 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9379bd2f-e4d3-42e1-ac64-1f74de859cf9 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:33.835576 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9379bd2f-e4d3-42e1-ac64-1f74de859cf9 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:33.836055 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9379bd2f-e4d3-42e1-ac64-1f74de859cf9 None admin] PATH_INFO: `/v3/users` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:33.837569 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9379bd2f-e4d3-42e1-ac64-1f74de859cf9 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:04:33.838053 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9379bd2f-e4d3-42e1-ac64-1f74de859cf9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:33.839891 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9379bd2f-e4d3-42e1-ac64-1f74de859cf9 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:33.859870 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 324/665] 10.222.0.20 () {56 vars in 1139 bytes} [Wed Aug 31 01:04:33 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:04:33.860712 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9379bd2f-e4d3-42e1-ac64-1f74de859cf9 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:33.918767 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dbfbbc-51b1-40e2-8324-d334de5c9dce None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:33.921917 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:33.955685 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04170c5b-2e48-47b5-ba16-64755ca6e930 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:33.956619 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04170c5b-2e48-47b5-ba16-64755ca6e930 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:33.957020 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04170c5b-2e48-47b5-ba16-64755ca6e930 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:33.957304 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04170c5b-2e48-47b5-ba16-64755ca6e930 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:33.966080 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04170c5b-2e48-47b5-ba16-64755ca6e930 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:33.967565 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04170c5b-2e48-47b5-ba16-64755ca6e930 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:33.980146 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application [None req-04170c5b-2e48-47b5-ba16-64755ca6e930 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 01:04:33.980146 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:04:33.980146 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:04:33.980146 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:04:33.980146 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:04:33.980146 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:04:33.980146 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:04:33.980146 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:04:33.980146 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:04:33.980146 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:04:33.980146 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:04:33.980146 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:04:33.980146 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 01:04:33.980146 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 01:04:33.980146 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 01:04:33.981296 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 01:04:33.981296 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:04:33.981296 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:04:33.981296 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:04:33.981296 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:04:33.981296 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:04:33.981296 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:04:33.981296 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:04:33.981296 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application with Lock( Aug 31 01:04:33.981296 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:04:33.981296 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:04:33.981296 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:04:33.981296 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:04:33.981296 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:04:33.981296 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:04:33.981296 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:04:33.981296 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:04:33.982182 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 01:04:33.982182 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 01:04:33.982182 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 01:04:33.982182 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 01:04:33.982182 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 01:04:33.982182 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 01:04:33.982182 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 01:04:33.982182 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Aug 31 01:04:33.982182 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 342/666] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Aug 31 01:04:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:04:33.982182 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04170c5b-2e48-47b5-ba16-64755ca6e930 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:33.991188 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9379bd2f-e4d3-42e1-ac64-1f74de859cf9 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:34.031128 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7764338-e37a-4a4b-8362-8adf86ec0948 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:34.031731 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7764338-e37a-4a4b-8362-8adf86ec0948 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:34.032018 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7764338-e37a-4a4b-8362-8adf86ec0948 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:34.032294 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7764338-e37a-4a4b-8362-8adf86ec0948 None admin] PATH_INFO: `/v3/domains` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:34.032971 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7764338-e37a-4a4b-8362-8adf86ec0948 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:04:34.033275 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7764338-e37a-4a4b-8362-8adf86ec0948 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:34.034655 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7764338-e37a-4a4b-8362-8adf86ec0948 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:34.044923 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:34.047485 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 325/667] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 31 01:04:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:04:34.048436 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7764338-e37a-4a4b-8362-8adf86ec0948 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:34.095011 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04170c5b-2e48-47b5-ba16-64755ca6e930 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:34.130983 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433e623c-72a4-4ea2-9b3b-afe3c9953eec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:34.131618 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433e623c-72a4-4ea2-9b3b-afe3c9953eec None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:34.131902 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433e623c-72a4-4ea2-9b3b-afe3c9953eec None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:34.132178 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433e623c-72a4-4ea2-9b3b-afe3c9953eec None admin] PATH_INFO: `/v3/roles/service` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:34.143550 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433e623c-72a4-4ea2-9b3b-afe3c9953eec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 01:04:34.151232 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application [None req-433e623c-72a4-4ea2-9b3b-afe3c9953eec None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 01:04:34.151232 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:04:34.151232 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:04:34.151232 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:04:34.151232 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:04:34.151232 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:04:34.151232 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:04:34.151232 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:04:34.151232 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:04:34.151232 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:04:34.151232 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:04:34.151232 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:04:34.151232 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 01:04:34.151232 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 01:04:34.151232 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 01:04:34.151232 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 01:04:34.152191 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 01:04:34.152191 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 01:04:34.152191 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 01:04:34.152191 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 01:04:34.152191 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:04:34.152191 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:04:34.152191 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:04:34.152191 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:04:34.152191 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:04:34.152191 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:04:34.152191 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:04:34.152191 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application with Lock( Aug 31 01:04:34.152191 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:04:34.152191 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:04:34.152191 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:04:34.152191 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:04:34.157613 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:04:34.157613 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:04:34.157613 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:04:34.157613 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:04:34.157613 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 01:04:34.157613 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 01:04:34.157613 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 01:04:34.157613 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 01:04:34.157613 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 01:04:34.157613 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 01:04:34.157613 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 01:04:34.157613 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Aug 31 01:04:34.157613 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 343/668] 10.222.0.20 () {56 vars in 1102 bytes} [Wed Aug 31 01:04:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:04:34.157613 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433e623c-72a4-4ea2-9b3b-afe3c9953eec None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:34.158556 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7764338-e37a-4a4b-8362-8adf86ec0948 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:34.164936 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:34.193915 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79efa900-bab8-4bc6-8e93-72a4567810c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:34.194923 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79efa900-bab8-4bc6-8e93-72a4567810c5 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:34.195505 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79efa900-bab8-4bc6-8e93-72a4567810c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:34.196144 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79efa900-bab8-4bc6-8e93-72a4567810c5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:34.264492 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433e623c-72a4-4ea2-9b3b-afe3c9953eec None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:34.268179 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79efa900-bab8-4bc6-8e93-72a4567810c5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:34.269698 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79efa900-bab8-4bc6-8e93-72a4567810c5 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:34.287753 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:34.307173 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14aa820f-6b50-43b1-80f9-c40f43c08a2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:34.307921 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14aa820f-6b50-43b1-80f9-c40f43c08a2f None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:34.308305 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14aa820f-6b50-43b1-80f9-c40f43c08a2f None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:34.308487 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14aa820f-6b50-43b1-80f9-c40f43c08a2f None admin] PATH_INFO: `/v3/projects` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:34.309215 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14aa820f-6b50-43b1-80f9-c40f43c08a2f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:04:34.309529 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14aa820f-6b50-43b1-80f9-c40f43c08a2f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:34.310964 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14aa820f-6b50-43b1-80f9-c40f43c08a2f None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:34.328755 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 344/669] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Aug 31 01:04:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:04:34.328755 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14aa820f-6b50-43b1-80f9-c40f43c08a2f None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:34.385762 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 326/670] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Aug 31 01:04:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 01:04:34.386659 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79efa900-bab8-4bc6-8e93-72a4567810c5 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:34.395472 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:34.434102 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14aa820f-6b50-43b1-80f9-c40f43c08a2f None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:34.470248 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55aadfe2-3094-42f0-acbd-e8244f100c7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:34.470856 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55aadfe2-3094-42f0-acbd-e8244f100c7a None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:34.471079 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55aadfe2-3094-42f0-acbd-e8244f100c7a None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:34.471354 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55aadfe2-3094-42f0-acbd-e8244f100c7a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:34.479543 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55aadfe2-3094-42f0-acbd-e8244f100c7a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:34.481480 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55aadfe2-3094-42f0-acbd-e8244f100c7a None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:34.492276 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79efa900-bab8-4bc6-8e93-72a4567810c5 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:34.495966 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application [None req-55aadfe2-3094-42f0-acbd-e8244f100c7a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 01:04:34.495966 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:04:34.495966 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:04:34.495966 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:04:34.495966 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:04:34.495966 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:04:34.495966 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:04:34.495966 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:04:34.495966 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:04:34.495966 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:04:34.495966 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:04:34.495966 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:04:34.495966 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 01:04:34.495966 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 01:04:34.495966 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 01:04:34.496878 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 01:04:34.496878 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:04:34.496878 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:04:34.496878 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:04:34.496878 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:04:34.496878 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:04:34.496878 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:04:34.496878 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:04:34.496878 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application with Lock( Aug 31 01:04:34.496878 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:04:34.496878 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:04:34.496878 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:04:34.496878 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:04:34.496878 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:04:34.496878 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:04:34.496878 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:04:34.496878 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:04:34.497756 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 01:04:34.497756 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 01:04:34.497756 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 01:04:34.497756 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 01:04:34.497756 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 01:04:34.497756 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 01:04:34.497756 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 01:04:34.497756 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Aug 31 01:04:34.497756 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 345/671] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Aug 31 01:04:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:04:34.498272 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55aadfe2-3094-42f0-acbd-e8244f100c7a None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:34.532613 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1ea9cb-a663-4ca6-aa8b-9996af9e6b05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:34.533576 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1ea9cb-a663-4ca6-aa8b-9996af9e6b05 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:34.533861 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1ea9cb-a663-4ca6-aa8b-9996af9e6b05 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:34.534137 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1ea9cb-a663-4ca6-aa8b-9996af9e6b05 None admin] PATH_INFO: `/v3/roles` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:34.534803 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1ea9cb-a663-4ca6-aa8b-9996af9e6b05 None admin] RBAC: Adding query filter params (name=service) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:04:34.535111 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1ea9cb-a663-4ca6-aa8b-9996af9e6b05 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:34.536487 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1ea9cb-a663-4ca6-aa8b-9996af9e6b05 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:34.545050 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 327/672] 10.222.0.20 () {56 vars in 1103 bytes} [Wed Aug 31 01:04:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:04:34.545841 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1ea9cb-a663-4ca6-aa8b-9996af9e6b05 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:34.549532 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:34.615827 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55aadfe2-3094-42f0-acbd-e8244f100c7a None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:34.618973 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:34.652369 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4161a8-dca4-436e-a584-31fc4de4de1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:34.652687 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1ea9cb-a663-4ca6-aa8b-9996af9e6b05 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:34.653051 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4161a8-dca4-436e-a584-31fc4de4de1a None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:34.653335 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4161a8-dca4-436e-a584-31fc4de4de1a None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:34.653612 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4161a8-dca4-436e-a584-31fc4de4de1a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:34.654337 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4161a8-dca4-436e-a584-31fc4de4de1a None admin] RBAC: Adding query filter params (role.id=7d00a4cfa13744a69604ac2cf1a5816e, scope.project.id=60b2abc15c2849ba8a1ed92117cc981b, user.id=d8590c288028418ebe7a4163fd8039f1) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:04:34.654652 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4161a8-dca4-436e-a584-31fc4de4de1a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:34.656159 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4161a8-dca4-436e-a584-31fc4de4de1a None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:34.662270 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 346/673] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Aug 31 01:04:34 2022] GET /identity/v3/role_assignments?user.id=d8590c288028418ebe7a4163fd8039f1&scope.project.id=60b2abc15c2849ba8a1ed92117cc981b&role.id=7d00a4cfa13744a69604ac2cf1a5816e => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:04:34.663065 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4161a8-dca4-436e-a584-31fc4de4de1a None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:34.703320 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b33045-3648-4887-8584-e1461aa4f6fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:34.704407 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b33045-3648-4887-8584-e1461aa4f6fa None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:34.704698 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b33045-3648-4887-8584-e1461aa4f6fa None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:34.705038 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b33045-3648-4887-8584-e1461aa4f6fa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:34.718325 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b33045-3648-4887-8584-e1461aa4f6fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:34.719862 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b33045-3648-4887-8584-e1461aa4f6fa None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:34.736939 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application [None req-06b33045-3648-4887-8584-e1461aa4f6fa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 01:04:34.736939 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:04:34.736939 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:04:34.736939 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:04:34.736939 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:04:34.736939 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:04:34.736939 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:04:34.736939 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:04:34.736939 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:04:34.736939 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:04:34.736939 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:04:34.736939 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:04:34.736939 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 01:04:34.736939 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 01:04:34.736939 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 01:04:34.736939 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 01:04:34.737925 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:04:34.737925 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:04:34.737925 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:04:34.737925 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:04:34.737925 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:04:34.737925 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:04:34.737925 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:04:34.737925 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application with Lock( Aug 31 01:04:34.737925 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:04:34.737925 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:04:34.737925 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:04:34.737925 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:04:34.737925 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:04:34.737925 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:04:34.737925 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:04:34.737925 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:04:34.737925 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 01:04:34.738864 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 01:04:34.738864 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 01:04:34.738864 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Aug 31 01:04:34.745582 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 328/674] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 01:04:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:04:34.746841 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b33045-3648-4887-8584-e1461aa4f6fa None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:34.786612 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4161a8-dca4-436e-a584-31fc4de4de1a None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:34.788361 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:34.826888 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e6e26e-589f-4056-b37b-0cd2c9d1fb85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:34.827520 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e6e26e-589f-4056-b37b-0cd2c9d1fb85 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:34.828078 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e6e26e-589f-4056-b37b-0cd2c9d1fb85 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:34.828392 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e6e26e-589f-4056-b37b-0cd2c9d1fb85 None admin] PATH_INFO: `/v3/projects` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:34.829382 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e6e26e-589f-4056-b37b-0cd2c9d1fb85 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:04:34.829698 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e6e26e-589f-4056-b37b-0cd2c9d1fb85 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:34.832141 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e6e26e-589f-4056-b37b-0cd2c9d1fb85 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:34.849043 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 347/675] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Aug 31 01:04:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:04:34.849878 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e6e26e-589f-4056-b37b-0cd2c9d1fb85 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:34.873255 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b33045-3648-4887-8584-e1461aa4f6fa None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:34.903654 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:34.921881 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148362e7-98d7-48a6-9cf7-adacae92fccf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:34.922681 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148362e7-98d7-48a6-9cf7-adacae92fccf None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:34.923294 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148362e7-98d7-48a6-9cf7-adacae92fccf None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:34.923660 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148362e7-98d7-48a6-9cf7-adacae92fccf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:34.936300 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148362e7-98d7-48a6-9cf7-adacae92fccf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:34.938748 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148362e7-98d7-48a6-9cf7-adacae92fccf None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:34.956933 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application [None req-148362e7-98d7-48a6-9cf7-adacae92fccf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 01:04:34.956933 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:04:34.956933 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:04:34.956933 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:04:34.956933 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:04:34.956933 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:04:34.956933 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:04:34.956933 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:04:34.956933 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:04:34.956933 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:04:34.956933 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:04:34.956933 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:04:34.956933 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 01:04:34.956933 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 01:04:34.956933 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 01:04:34.956933 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 01:04:34.957892 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:04:34.957892 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:04:34.957892 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:04:34.957892 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:04:34.957892 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:04:34.957892 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:04:34.957892 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:04:34.957892 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application with Lock( Aug 31 01:04:34.957892 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:04:34.957892 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:04:34.957892 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:04:34.957892 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:04:34.957892 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:04:34.957892 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:04:34.957892 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:04:34.957892 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:04:34.957892 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 01:04:34.958781 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 01:04:34.958781 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 01:04:34.958781 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Aug 31 01:04:34.958781 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 329/676] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 01:04:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:04:34.959414 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dd4f85-0594-4103-b467-536e420a44a1 None None] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:34.959521 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dd4f85-0594-4103-b467-536e420a44a1 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:34.959779 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dd4f85-0594-4103-b467-536e420a44a1 None None] PATH_INFO: `/` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:34.961847 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 330/677] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:04:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:04:34.962568 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dd4f85-0594-4103-b467-536e420a44a1 None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:34.968532 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e6e26e-589f-4056-b37b-0cd2c9d1fb85 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:35.003775 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0981b4c-b08c-4370-9a26-e286706865b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:35.004469 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0981b4c-b08c-4370-9a26-e286706865b8 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:35.004898 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0981b4c-b08c-4370-9a26-e286706865b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:35.005274 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0981b4c-b08c-4370-9a26-e286706865b8 None admin] PATH_INFO: `/v3/domains` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:35.006023 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0981b4c-b08c-4370-9a26-e286706865b8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:04:35.006460 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0981b4c-b08c-4370-9a26-e286706865b8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:35.008440 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0981b4c-b08c-4370-9a26-e286706865b8 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:35.021216 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 348/678] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 31 01:04:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:04:35.022557 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198144d9-64bc-4e50-8442-80361526ce2e None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:35.023479 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198144d9-64bc-4e50-8442-80361526ce2e None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:35.023881 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198144d9-64bc-4e50-8442-80361526ce2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:35.044532 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-198144d9-64bc-4e50-8442-80361526ce2e None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:04:35.046062 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-198144d9-64bc-4e50-8442-80361526ce2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:04:35.065945 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:35.083523 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dd4f85-0594-4103-b467-536e420a44a1 None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:35.118396 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 349/679] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:04:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 01:04:35.119354 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8808e99e-4c83-4427-ae47-1110c10c9a05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:35.119962 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8808e99e-4c83-4427-ae47-1110c10c9a05 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:35.120246 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8808e99e-4c83-4427-ae47-1110c10c9a05 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:35.120526 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8808e99e-4c83-4427-ae47-1110c10c9a05 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:35.121287 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8808e99e-4c83-4427-ae47-1110c10c9a05 None admin] RBAC: Adding query filter params (role.id=7d00a4cfa13744a69604ac2cf1a5816e, scope.project.id=60b2abc15c2849ba8a1ed92117cc981b, user.id=a5c8fc69c518412ba1aad61a9f4f558c) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:04:35.121599 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8808e99e-4c83-4427-ae47-1110c10c9a05 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:35.123034 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8808e99e-4c83-4427-ae47-1110c10c9a05 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:35.137128 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 331/680] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Aug 31 01:04:34 2022] GET /identity/v3/role_assignments?user.id=a5c8fc69c518412ba1aad61a9f4f558c&scope.project.id=60b2abc15c2849ba8a1ed92117cc981b&role.id=7d00a4cfa13744a69604ac2cf1a5816e => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:04:35.137321 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198144d9-64bc-4e50-8442-80361526ce2e None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:35.137428 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8808e99e-4c83-4427-ae47-1110c10c9a05 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:35.217121 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:35.328237 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8808e99e-4c83-4427-ae47-1110c10c9a05 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:35.329920 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198144d9-64bc-4e50-8442-80361526ce2e None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:35.364982 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0867719-5883-4a26-9a29-d139e05ee901 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:35.365719 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0867719-5883-4a26-9a29-d139e05ee901 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:35.365953 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0867719-5883-4a26-9a29-d139e05ee901 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:35.366229 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0867719-5883-4a26-9a29-d139e05ee901 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:35.370701 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d43780-d424-48d1-a82d-f732a3f3efbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:35.371563 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d43780-d424-48d1-a82d-f732a3f3efbf None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:35.372089 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d43780-d424-48d1-a82d-f732a3f3efbf None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:35.372565 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d43780-d424-48d1-a82d-f732a3f3efbf None admin] PATH_INFO: `/v3/domains` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:35.373428 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d43780-d424-48d1-a82d-f732a3f3efbf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:04:35.373938 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d43780-d424-48d1-a82d-f732a3f3efbf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:35.374043 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0867719-5883-4a26-9a29-d139e05ee901 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:35.375434 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0867719-5883-4a26-9a29-d139e05ee901 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:35.376028 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d43780-d424-48d1-a82d-f732a3f3efbf None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:35.388595 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application [None req-a0867719-5883-4a26-9a29-d139e05ee901 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 01:04:35.388595 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:04:35.388595 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:04:35.388595 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:04:35.388595 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:04:35.388595 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:04:35.388595 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:04:35.388595 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:04:35.388595 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:04:35.388595 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:04:35.388595 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:04:35.388595 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:04:35.388595 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 01:04:35.388595 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 01:04:35.388595 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 01:04:35.389664 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 01:04:35.389664 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:04:35.389664 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:04:35.389664 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:04:35.389664 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:04:35.389664 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:04:35.389664 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:04:35.389664 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:04:35.389664 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application with Lock( Aug 31 01:04:35.389664 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:04:35.389664 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:04:35.389664 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:04:35.389664 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:04:35.389664 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:04:35.389664 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:04:35.389664 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:04:35.389664 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:04:35.390561 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 01:04:35.390561 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 01:04:35.390561 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 01:04:35.390561 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 01:04:35.390561 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 01:04:35.390561 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 01:04:35.390561 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 01:04:35.390561 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Aug 31 01:04:35.390561 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 332/681] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Aug 31 01:04:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:04:35.396200 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0867719-5883-4a26-9a29-d139e05ee901 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:35.396654 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 350/682] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 31 01:04:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:04:35.397717 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d43780-d424-48d1-a82d-f732a3f3efbf None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:35.415765 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:35.499784 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:35.505779 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0867719-5883-4a26-9a29-d139e05ee901 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:35.525912 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d43780-d424-48d1-a82d-f732a3f3efbf None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:35.541837 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f34add-3a10-4591-aefe-6042285f4fab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:35.542730 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f34add-3a10-4591-aefe-6042285f4fab None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:35.543115 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f34add-3a10-4591-aefe-6042285f4fab None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:35.543508 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f34add-3a10-4591-aefe-6042285f4fab None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:35.555203 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f34add-3a10-4591-aefe-6042285f4fab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 01:04:35.561551 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application [None req-e2f34add-3a10-4591-aefe-6042285f4fab None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 01:04:35.561551 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:04:35.561551 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:04:35.561551 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:04:35.561551 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:04:35.561551 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:04:35.561551 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:04:35.561551 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:04:35.561551 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:04:35.561551 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:04:35.561551 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:04:35.561551 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:04:35.561551 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 01:04:35.561551 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 01:04:35.561551 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 01:04:35.561551 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 01:04:35.562507 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 01:04:35.562507 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 01:04:35.562507 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 01:04:35.562507 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 01:04:35.562507 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:04:35.562507 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:04:35.562507 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:04:35.562507 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:04:35.562507 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:04:35.562507 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:04:35.562507 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:04:35.562507 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application with Lock( Aug 31 01:04:35.562507 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:04:35.562507 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:04:35.562507 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:04:35.562507 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:04:35.568401 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:04:35.568401 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:04:35.568401 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:04:35.568401 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:04:35.568401 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 01:04:35.568401 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 01:04:35.568401 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 01:04:35.568401 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 01:04:35.568401 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 01:04:35.568401 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 01:04:35.568401 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 01:04:35.568401 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Aug 31 01:04:35.568401 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 333/683] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Aug 31 01:04:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:04:35.568401 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f34add-3a10-4591-aefe-6042285f4fab None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:35.579064 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c3897d-b5b9-4879-9f18-63ffd4dc216b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:35.579619 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c3897d-b5b9-4879-9f18-63ffd4dc216b None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:35.579900 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c3897d-b5b9-4879-9f18-63ffd4dc216b None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:35.580178 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c3897d-b5b9-4879-9f18-63ffd4dc216b None admin] PATH_INFO: `/v3/projects` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:35.580909 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c3897d-b5b9-4879-9f18-63ffd4dc216b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:04:35.581215 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c3897d-b5b9-4879-9f18-63ffd4dc216b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:35.582672 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c3897d-b5b9-4879-9f18-63ffd4dc216b None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:35.597658 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 351/684] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Aug 31 01:04:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:04:35.598486 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c3897d-b5b9-4879-9f18-63ffd4dc216b None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:35.681543 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:35.698881 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f34add-3a10-4591-aefe-6042285f4fab None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:35.712908 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c3897d-b5b9-4879-9f18-63ffd4dc216b None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:35.745968 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4dbde0-4084-492a-921b-f1e4b396ba04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:35.747508 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4dbde0-4084-492a-921b-f1e4b396ba04 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:35.748000 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4dbde0-4084-492a-921b-f1e4b396ba04 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:35.748680 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4dbde0-4084-492a-921b-f1e4b396ba04 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:35.755648 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cc40a4-4660-4d54-aa87-d9b6a97ed172 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:35.756262 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cc40a4-4660-4d54-aa87-d9b6a97ed172 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:35.756545 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cc40a4-4660-4d54-aa87-d9b6a97ed172 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:35.756852 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cc40a4-4660-4d54-aa87-d9b6a97ed172 None admin] PATH_INFO: `/v3/roles` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:35.757555 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cc40a4-4660-4d54-aa87-d9b6a97ed172 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:04:35.757861 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cc40a4-4660-4d54-aa87-d9b6a97ed172 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:35.759264 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cc40a4-4660-4d54-aa87-d9b6a97ed172 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:35.759632 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4dbde0-4084-492a-921b-f1e4b396ba04 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:35.761382 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4dbde0-4084-492a-921b-f1e4b396ba04 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:35.770080 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 352/685] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Aug 31 01:04:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:04:35.771240 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cc40a4-4660-4d54-aa87-d9b6a97ed172 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:35.780464 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application [None req-bf4dbde0-4084-492a-921b-f1e4b396ba04 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 31 01:04:35.780464 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:04:35.780464 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:04:35.780464 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:04:35.780464 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:04:35.780464 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:04:35.780464 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:04:35.780464 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:04:35.780464 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:04:35.780464 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:04:35.780464 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:04:35.780464 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:04:35.780464 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 01:04:35.780464 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 01:04:35.780464 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 01:04:35.780464 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 01:04:35.781472 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:04:35.781472 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:04:35.781472 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 01:04:35.781472 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 01:04:35.781472 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 01:04:35.781472 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 01:04:35.781472 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:04:35.781472 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:04:35.781472 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:04:35.781472 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:04:35.781472 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:04:35.781472 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application with Lock( Aug 31 01:04:35.781472 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:04:35.781472 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:04:35.781472 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:04:35.781472 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:04:35.781472 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:04:35.781472 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:04:35.782446 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:04:35.782446 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:04:35.782446 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 01:04:35.782446 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 01:04:35.782446 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 01:04:35.782446 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 01:04:35.782446 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 01:04:35.782446 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 01:04:35.782446 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 31 01:04:35.782446 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Aug 31 01:04:35.784798 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 334/686] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Aug 31 01:04:35 2022] GET /identity/v3/users/swift => generated 83 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:04:35.785948 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4dbde0-4084-492a-921b-f1e4b396ba04 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:35.811021 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:35.894497 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cc40a4-4660-4d54-aa87-d9b6a97ed172 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:35.902062 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4dbde0-4084-492a-921b-f1e4b396ba04 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:35.934445 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e563a5a-ca3d-46d1-ad95-ae9cf8c86017 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:35.935003 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e563a5a-ca3d-46d1-ad95-ae9cf8c86017 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:35.935285 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e563a5a-ca3d-46d1-ad95-ae9cf8c86017 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:35.935556 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e563a5a-ca3d-46d1-ad95-ae9cf8c86017 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:35.936382 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e563a5a-ca3d-46d1-ad95-ae9cf8c86017 None admin] RBAC: Adding query filter params (role.id=7d00a4cfa13744a69604ac2cf1a5816e, scope.project.id=60b2abc15c2849ba8a1ed92117cc981b, user.id=0817c3320d5d41d8bba0550c8ae8dd60) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:04:35.936692 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e563a5a-ca3d-46d1-ad95-ae9cf8c86017 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:35.938276 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e563a5a-ca3d-46d1-ad95-ae9cf8c86017 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:35.939908 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05640f5-f19d-48c4-8711-6c71be821fbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:35.940170 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:35.941003 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05640f5-f19d-48c4-8711-6c71be821fbe None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:35.941518 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05640f5-f19d-48c4-8711-6c71be821fbe None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:35.941994 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05640f5-f19d-48c4-8711-6c71be821fbe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:35.945439 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 353/687] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Aug 31 01:04:35 2022] GET /identity/v3/role_assignments?user.id=0817c3320d5d41d8bba0550c8ae8dd60&scope.project.id=60b2abc15c2849ba8a1ed92117cc981b&role.id=7d00a4cfa13744a69604ac2cf1a5816e => generated 608 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:04:35.946303 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e563a5a-ca3d-46d1-ad95-ae9cf8c86017 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:35.954692 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05640f5-f19d-48c4-8711-6c71be821fbe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:35.959911 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05640f5-f19d-48c4-8711-6c71be821fbe None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:35.973088 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application [None req-b05640f5-f19d-48c4-8711-6c71be821fbe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 01:04:35.973088 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:04:35.973088 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:04:35.973088 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:04:35.973088 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:04:35.973088 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:04:35.973088 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:04:35.973088 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:04:35.973088 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:04:35.973088 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:04:35.973088 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:04:35.973088 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:04:35.973088 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 01:04:35.973088 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 01:04:35.973088 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 01:04:35.973088 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 01:04:35.974107 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:04:35.974107 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:04:35.974107 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:04:35.974107 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:04:35.974107 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:04:35.974107 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:04:35.974107 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:04:35.974107 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application with Lock( Aug 31 01:04:35.974107 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:04:35.974107 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:04:35.974107 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:04:35.974107 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:04:35.974107 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:04:35.974107 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:04:35.974107 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:04:35.974107 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:04:35.974107 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 01:04:35.975007 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 01:04:35.975007 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 01:04:35.975007 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Aug 31 01:04:35.977784 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 335/688] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 01:04:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:04:35.983570 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05640f5-f19d-48c4-8711-6c71be821fbe None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:36.062411 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e563a5a-ca3d-46d1-ad95-ae9cf8c86017 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:36.078887 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:36.108822 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95122a43-1f4b-4c98-a6b0-5ed1d4287fbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:36.109681 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95122a43-1f4b-4c98-a6b0-5ed1d4287fbd None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:36.111803 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95122a43-1f4b-4c98-a6b0-5ed1d4287fbd None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:36.112252 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95122a43-1f4b-4c98-a6b0-5ed1d4287fbd None admin] PATH_INFO: `/v3/users` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:36.113706 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95122a43-1f4b-4c98-a6b0-5ed1d4287fbd None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:04:36.114995 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95122a43-1f4b-4c98-a6b0-5ed1d4287fbd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:36.117721 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95122a43-1f4b-4c98-a6b0-5ed1d4287fbd None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:36.140002 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05640f5-f19d-48c4-8711-6c71be821fbe None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:36.140857 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 354/689] 10.222.0.20 () {56 vars in 1135 bytes} [Wed Aug 31 01:04:35 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:04:36.143166 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95122a43-1f4b-4c98-a6b0-5ed1d4287fbd None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:36.185724 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ba3257-a15c-4426-b3b1-33356ab3c02c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:36.186355 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ba3257-a15c-4426-b3b1-33356ab3c02c None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:36.186637 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ba3257-a15c-4426-b3b1-33356ab3c02c None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:36.186907 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ba3257-a15c-4426-b3b1-33356ab3c02c None admin] PATH_INFO: `/v3/domains` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:36.187617 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ba3257-a15c-4426-b3b1-33356ab3c02c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:04:36.187926 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ba3257-a15c-4426-b3b1-33356ab3c02c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:36.189396 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ba3257-a15c-4426-b3b1-33356ab3c02c None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:36.204293 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 336/690] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 31 01:04:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 01:04:36.209440 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ba3257-a15c-4426-b3b1-33356ab3c02c None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:36.210159 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:36.260095 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95122a43-1f4b-4c98-a6b0-5ed1d4287fbd None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:36.299273 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8349e34-71c9-474b-81d9-379d67f7d2b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:36.299886 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8349e34-71c9-474b-81d9-379d67f7d2b2 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:36.300172 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8349e34-71c9-474b-81d9-379d67f7d2b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:36.300449 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8349e34-71c9-474b-81d9-379d67f7d2b2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:36.317796 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ba3257-a15c-4426-b3b1-33356ab3c02c None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:36.347751 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:36.356864 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a78fd9-0ba0-4660-a653-8b20052fd28a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:36.373873 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a78fd9-0ba0-4660-a653-8b20052fd28a None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:36.373873 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a78fd9-0ba0-4660-a653-8b20052fd28a None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:36.373873 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a78fd9-0ba0-4660-a653-8b20052fd28a None admin] PATH_INFO: `/v3/users/nova` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:36.373873 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a78fd9-0ba0-4660-a653-8b20052fd28a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:36.373873 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a78fd9-0ba0-4660-a653-8b20052fd28a None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:36.374414 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8349e34-71c9-474b-81d9-379d67f7d2b2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:36.375829 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8349e34-71c9-474b-81d9-379d67f7d2b2 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:36.379600 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application [None req-66a78fd9-0ba0-4660-a653-8b20052fd28a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 31 01:04:36.379600 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:04:36.379600 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:04:36.379600 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:04:36.379600 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:04:36.379600 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:04:36.379600 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:04:36.379600 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:04:36.379600 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:04:36.379600 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:04:36.379600 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:04:36.379600 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:04:36.379600 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 01:04:36.379600 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 01:04:36.379600 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 01:04:36.379600 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 01:04:36.380611 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:04:36.380611 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:04:36.380611 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 01:04:36.380611 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 01:04:36.380611 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 01:04:36.380611 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 01:04:36.380611 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:04:36.380611 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:04:36.380611 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:04:36.380611 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:04:36.380611 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:04:36.380611 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application with Lock( Aug 31 01:04:36.380611 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:04:36.380611 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:04:36.380611 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:04:36.380611 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:04:36.380611 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:04:36.380611 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:04:36.381589 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:04:36.381589 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:04:36.381589 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 01:04:36.381589 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 01:04:36.381589 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 01:04:36.381589 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 01:04:36.381589 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 01:04:36.381589 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 01:04:36.381589 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 31 01:04:36.381589 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Aug 31 01:04:36.381589 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 337/691] 10.222.0.20 () {56 vars in 1093 bytes} [Wed Aug 31 01:04:36 2022] GET /identity/v3/users/nova => generated 82 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:04:36.393751 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a78fd9-0ba0-4660-a653-8b20052fd28a None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:36.454311 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:36.527975 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 355/692] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Aug 31 01:04:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 31 01:04:36.533475 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8349e34-71c9-474b-81d9-379d67f7d2b2 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:36.594472 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a78fd9-0ba0-4660-a653-8b20052fd28a None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:36.638309 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8349e34-71c9-474b-81d9-379d67f7d2b2 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:36.655833 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a6e657-29b4-4ed0-b0f8-03e44cef7ef1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:36.656537 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a6e657-29b4-4ed0-b0f8-03e44cef7ef1 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:36.656856 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a6e657-29b4-4ed0-b0f8-03e44cef7ef1 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:36.657224 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a6e657-29b4-4ed0-b0f8-03e44cef7ef1 None admin] PATH_INFO: `/v3/users` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:36.657962 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a6e657-29b4-4ed0-b0f8-03e44cef7ef1 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:04:36.658267 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a6e657-29b4-4ed0-b0f8-03e44cef7ef1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:36.659805 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a6e657-29b4-4ed0-b0f8-03e44cef7ef1 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:36.674124 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdcf5bd-bc9a-4057-8fec-be1ad1a4a558 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:36.674792 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdcf5bd-bc9a-4057-8fec-be1ad1a4a558 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:36.675167 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdcf5bd-bc9a-4057-8fec-be1ad1a4a558 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:36.675445 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdcf5bd-bc9a-4057-8fec-be1ad1a4a558 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:36.685850 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdcf5bd-bc9a-4057-8fec-be1ad1a4a558 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:36.687430 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdcf5bd-bc9a-4057-8fec-be1ad1a4a558 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:36.693284 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 338/693] 10.222.0.20 () {56 vars in 1133 bytes} [Wed Aug 31 01:04:36 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:04:36.708532 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application [None req-7cdcf5bd-bc9a-4057-8fec-be1ad1a4a558 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 01:04:36.708532 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:04:36.708532 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:04:36.708532 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:04:36.708532 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:04:36.708532 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:04:36.708532 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:04:36.708532 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:04:36.708532 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:04:36.708532 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:04:36.708532 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:04:36.708532 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:04:36.708532 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 01:04:36.708532 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 01:04:36.708532 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 01:04:36.708532 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 01:04:36.711419 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:04:36.711419 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:04:36.711419 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:04:36.711419 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:04:36.711419 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:04:36.711419 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:04:36.711419 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:04:36.711419 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application with Lock( Aug 31 01:04:36.711419 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:04:36.711419 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:04:36.711419 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:04:36.711419 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:04:36.711419 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:04:36.711419 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:04:36.711419 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:04:36.711419 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:04:36.711419 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 01:04:36.713610 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 01:04:36.713610 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 01:04:36.713610 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Aug 31 01:04:36.713989 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a6e657-29b4-4ed0-b0f8-03e44cef7ef1 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:36.714134 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 356/694] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 01:04:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 01:04:36.720747 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdcf5bd-bc9a-4057-8fec-be1ad1a4a558 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:36.729439 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:36.788307 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:36.825896 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdcf5bd-bc9a-4057-8fec-be1ad1a4a558 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:36.862024 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a23757-1fd2-4cf4-8cb0-e2d768f555f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:36.863215 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a23757-1fd2-4cf4-8cb0-e2d768f555f9 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:36.863215 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a23757-1fd2-4cf4-8cb0-e2d768f555f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:36.863630 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a23757-1fd2-4cf4-8cb0-e2d768f555f9 None admin] PATH_INFO: `/v3/domains` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:36.863935 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a23757-1fd2-4cf4-8cb0-e2d768f555f9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:04:36.864240 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a23757-1fd2-4cf4-8cb0-e2d768f555f9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:36.867116 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a23757-1fd2-4cf4-8cb0-e2d768f555f9 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:36.878455 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a6e657-29b4-4ed0-b0f8-03e44cef7ef1 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:36.879457 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 357/695] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 31 01:04:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:04:36.885755 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1e0bda-9103-4aad-9b17-5167cc5234e4 None None] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:36.886042 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1e0bda-9103-4aad-9b17-5167cc5234e4 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:36.886323 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1e0bda-9103-4aad-9b17-5167cc5234e4 None None] PATH_INFO: `/` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:36.891703 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 358/696] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:04:36 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:04:36.891703 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1e0bda-9103-4aad-9b17-5167cc5234e4 None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:36.923685 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5513ca-5a88-486c-8d3c-efdc83c0b673 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:36.925030 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5513ca-5a88-486c-8d3c-efdc83c0b673 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:36.925581 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5513ca-5a88-486c-8d3c-efdc83c0b673 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:36.926022 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5513ca-5a88-486c-8d3c-efdc83c0b673 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:36.989217 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:36.997243 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5513ca-5a88-486c-8d3c-efdc83c0b673 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:36.998915 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5513ca-5a88-486c-8d3c-efdc83c0b673 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:36.999989 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1e0bda-9103-4aad-9b17-5167cc5234e4 None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:37.037899 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc749a9c-a734-4498-a421-a14d68dfb8e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:37.038764 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc749a9c-a734-4498-a421-a14d68dfb8e5 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:37.039146 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc749a9c-a734-4498-a421-a14d68dfb8e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:37.039541 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc749a9c-a734-4498-a421-a14d68dfb8e5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:37.047577 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc749a9c-a734-4498-a421-a14d68dfb8e5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:37.049097 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:37.050788 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc749a9c-a734-4498-a421-a14d68dfb8e5 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:37.063561 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application [None req-cc749a9c-a734-4498-a421-a14d68dfb8e5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 01:04:37.063561 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:04:37.063561 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:04:37.063561 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:04:37.063561 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:04:37.063561 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:04:37.063561 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:04:37.063561 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:04:37.063561 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:04:37.063561 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:04:37.063561 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:04:37.063561 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:04:37.063561 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 01:04:37.063561 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 01:04:37.063561 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 01:04:37.065191 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 01:04:37.065191 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:04:37.065191 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:04:37.065191 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:04:37.065191 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:04:37.065191 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:04:37.065191 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:04:37.065191 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:04:37.065191 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application with Lock( Aug 31 01:04:37.065191 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:04:37.065191 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:04:37.065191 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:04:37.065191 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:04:37.065191 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:04:37.065191 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:04:37.065191 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:04:37.065191 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:04:37.066096 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 01:04:37.066096 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 01:04:37.066096 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 01:04:37.066096 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 01:04:37.066096 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 01:04:37.066096 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 01:04:37.066096 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 01:04:37.066096 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Aug 31 01:04:37.073001 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 359/697] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Aug 31 01:04:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:04:37.074744 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cea770-312c-4340-a25c-c3b34a001aa2 None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:37.075237 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cea770-312c-4340-a25c-c3b34a001aa2 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:37.075739 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cea770-312c-4340-a25c-c3b34a001aa2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:37.097908 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-29cea770-312c-4340-a25c-c3b34a001aa2 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:04:37.099434 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-29cea770-312c-4340-a25c-c3b34a001aa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:04:37.111950 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 339/698] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Aug 31 01:04:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 31 01:04:37.111950 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5513ca-5a88-486c-8d3c-efdc83c0b673 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:37.178268 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 360/699] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:04:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 01:04:37.180084 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cea770-312c-4340-a25c-c3b34a001aa2 None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:37.235325 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5513ca-5a88-486c-8d3c-efdc83c0b673 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:37.241508 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:37.286357 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cea770-312c-4340-a25c-c3b34a001aa2 None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:37.297688 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bafc58-5eaa-4a12-8ee4-184e30b3826d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:37.298280 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bafc58-5eaa-4a12-8ee4-184e30b3826d None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:37.298565 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bafc58-5eaa-4a12-8ee4-184e30b3826d None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:37.298840 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bafc58-5eaa-4a12-8ee4-184e30b3826d None admin] PATH_INFO: `/v3/projects` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:37.299531 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bafc58-5eaa-4a12-8ee4-184e30b3826d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:04:37.299836 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bafc58-5eaa-4a12-8ee4-184e30b3826d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:37.301421 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bafc58-5eaa-4a12-8ee4-184e30b3826d None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:37.322891 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 340/700] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Aug 31 01:04:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:04:37.329713 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bafc58-5eaa-4a12-8ee4-184e30b3826d None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:37.331463 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8f723b-8039-4277-8e76-8932993bca45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:37.332341 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8f723b-8039-4277-8e76-8932993bca45 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:37.332865 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8f723b-8039-4277-8e76-8932993bca45 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:37.333471 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8f723b-8039-4277-8e76-8932993bca45 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:37.344811 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8f723b-8039-4277-8e76-8932993bca45 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:37.346684 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8f723b-8039-4277-8e76-8932993bca45 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:37.350141 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:37.364559 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application [None req-de8f723b-8039-4277-8e76-8932993bca45 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 01:04:37.364559 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:04:37.364559 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:04:37.364559 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:04:37.364559 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:04:37.364559 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:04:37.364559 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:04:37.364559 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:04:37.364559 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:04:37.364559 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:04:37.364559 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:04:37.364559 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:04:37.364559 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 01:04:37.364559 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 01:04:37.364559 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 01:04:37.364559 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 01:04:37.365564 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:04:37.365564 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:04:37.365564 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:04:37.365564 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:04:37.365564 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:04:37.365564 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:04:37.365564 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:04:37.365564 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application with Lock( Aug 31 01:04:37.365564 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:04:37.365564 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:04:37.365564 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:04:37.365564 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:04:37.365564 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:04:37.365564 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:04:37.365564 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:04:37.365564 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:04:37.365564 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 01:04:37.366467 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 01:04:37.366467 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 01:04:37.366467 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Aug 31 01:04:37.369169 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 361/701] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 01:04:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:04:37.374792 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8f723b-8039-4277-8e76-8932993bca45 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:37.482093 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8f723b-8039-4277-8e76-8932993bca45 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:37.503487 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:37.519933 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f162b3-f0cc-408c-bb1b-7f234771b784 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:37.520938 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f162b3-f0cc-408c-bb1b-7f234771b784 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:37.521232 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f162b3-f0cc-408c-bb1b-7f234771b784 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:37.521511 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f162b3-f0cc-408c-bb1b-7f234771b784 None admin] PATH_INFO: `/v3/domains` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:37.522195 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f162b3-f0cc-408c-bb1b-7f234771b784 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:04:37.522507 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f162b3-f0cc-408c-bb1b-7f234771b784 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:37.525389 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f162b3-f0cc-408c-bb1b-7f234771b784 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:37.532039 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bafc58-5eaa-4a12-8ee4-184e30b3826d None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:37.543041 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 362/702] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 31 01:04:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 01:04:37.545176 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78447026-946a-486c-9258-f00b811f9fbd None None] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:37.545465 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78447026-946a-486c-9258-f00b811f9fbd None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:37.545735 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78447026-946a-486c-9258-f00b811f9fbd None None] PATH_INFO: `/` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:37.547193 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 363/703] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:04:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:04:37.548690 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78447026-946a-486c-9258-f00b811f9fbd None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:37.569972 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df3d9ad-818d-435c-a1e9-e7999949cd80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:37.570666 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df3d9ad-818d-435c-a1e9-e7999949cd80 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:37.570947 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df3d9ad-818d-435c-a1e9-e7999949cd80 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:37.571685 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df3d9ad-818d-435c-a1e9-e7999949cd80 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:37.572554 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df3d9ad-818d-435c-a1e9-e7999949cd80 None admin] RBAC: Adding query filter params (role.id=7d00a4cfa13744a69604ac2cf1a5816e, scope.project.id=60b2abc15c2849ba8a1ed92117cc981b, user.id=fc1d010919ec40878ae829cc8dceec4c) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:04:37.573016 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df3d9ad-818d-435c-a1e9-e7999949cd80 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:37.575208 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df3d9ad-818d-435c-a1e9-e7999949cd80 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:37.583102 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 341/704] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Aug 31 01:04:37 2022] GET /identity/v3/role_assignments?user.id=fc1d010919ec40878ae829cc8dceec4c&scope.project.id=60b2abc15c2849ba8a1ed92117cc981b&role.id=7d00a4cfa13744a69604ac2cf1a5816e => generated 608 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:04:37.583958 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df3d9ad-818d-435c-a1e9-e7999949cd80 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:37.681091 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:37.686127 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78447026-946a-486c-9258-f00b811f9fbd None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:37.724556 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df3d9ad-818d-435c-a1e9-e7999949cd80 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:37.731859 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761017c1-ff7b-474c-a4ae-ceb118a2e3ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:37.731859 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761017c1-ff7b-474c-a4ae-ceb118a2e3ce None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:37.731859 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761017c1-ff7b-474c-a4ae-ceb118a2e3ce None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:37.731859 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761017c1-ff7b-474c-a4ae-ceb118a2e3ce None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:37.741921 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761017c1-ff7b-474c-a4ae-ceb118a2e3ce None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:37.744575 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761017c1-ff7b-474c-a4ae-ceb118a2e3ce None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:37.761130 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application [None req-761017c1-ff7b-474c-a4ae-ceb118a2e3ce None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 01:04:37.761130 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:04:37.761130 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:04:37.761130 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:04:37.761130 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:04:37.761130 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:04:37.761130 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:04:37.761130 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:04:37.761130 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:04:37.761130 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:04:37.761130 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:04:37.761130 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:04:37.761130 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 01:04:37.761130 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 01:04:37.761130 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 01:04:37.761130 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 01:04:37.762127 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:04:37.762127 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:04:37.762127 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:04:37.762127 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:04:37.762127 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:04:37.762127 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:04:37.762127 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:04:37.762127 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application with Lock( Aug 31 01:04:37.762127 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:04:37.762127 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:04:37.762127 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:04:37.762127 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:04:37.762127 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:04:37.762127 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:04:37.762127 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:04:37.762127 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:04:37.762127 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 01:04:37.763018 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 01:04:37.763018 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 01:04:37.763018 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Aug 31 01:04:37.763018 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 364/705] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 01:04:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:04:37.765801 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849ac130-4b5a-41ad-a8d4-e08c2a332378 None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:37.765801 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849ac130-4b5a-41ad-a8d4-e08c2a332378 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:37.765801 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849ac130-4b5a-41ad-a8d4-e08c2a332378 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:37.775491 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5a78c8-e2b2-40d0-adc3-1487135109d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:37.776131 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5a78c8-e2b2-40d0-adc3-1487135109d9 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:37.776420 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5a78c8-e2b2-40d0-adc3-1487135109d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:37.776697 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5a78c8-e2b2-40d0-adc3-1487135109d9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:37.785686 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5a78c8-e2b2-40d0-adc3-1487135109d9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:37.787157 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5a78c8-e2b2-40d0-adc3-1487135109d9 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:37.790270 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-849ac130-4b5a-41ad-a8d4-e08c2a332378 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:04:37.791476 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-849ac130-4b5a-41ad-a8d4-e08c2a332378 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:04:37.802819 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application [None req-0d5a78c8-e2b2-40d0-adc3-1487135109d9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 01:04:37.802819 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:04:37.802819 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:04:37.802819 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:04:37.802819 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:04:37.802819 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:04:37.802819 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:04:37.802819 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:04:37.802819 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:04:37.802819 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:04:37.802819 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:04:37.802819 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:04:37.802819 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 01:04:37.802819 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 01:04:37.802819 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 01:04:37.803929 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 01:04:37.803929 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:04:37.803929 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:04:37.803929 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:04:37.803929 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:04:37.803929 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:04:37.803929 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:04:37.803929 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:04:37.803929 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application with Lock( Aug 31 01:04:37.803929 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:04:37.803929 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:04:37.803929 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:04:37.803929 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:04:37.803929 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:04:37.803929 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:04:37.803929 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:04:37.803929 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:04:37.805052 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 01:04:37.805052 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 01:04:37.805052 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 01:04:37.805052 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 01:04:37.805052 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 01:04:37.805052 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 01:04:37.805052 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 01:04:37.805052 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Aug 31 01:04:37.806287 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 342/706] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Aug 31 01:04:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:04:37.806938 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5a78c8-e2b2-40d0-adc3-1487135109d9 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:37.814634 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:37.878487 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 365/707] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:04:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 01:04:37.879499 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849ac130-4b5a-41ad-a8d4-e08c2a332378 None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:37.950330 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5a78c8-e2b2-40d0-adc3-1487135109d9 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:38.006808 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:38.019786 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89178c99-52e3-43fa-bb32-1dbb67f92299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:38.020643 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89178c99-52e3-43fa-bb32-1dbb67f92299 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:38.021091 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89178c99-52e3-43fa-bb32-1dbb67f92299 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:38.021547 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89178c99-52e3-43fa-bb32-1dbb67f92299 None admin] PATH_INFO: `/v3/domains` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:38.022348 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89178c99-52e3-43fa-bb32-1dbb67f92299 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:04:38.025088 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89178c99-52e3-43fa-bb32-1dbb67f92299 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:38.026641 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89178c99-52e3-43fa-bb32-1dbb67f92299 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:38.041417 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849ac130-4b5a-41ad-a8d4-e08c2a332378 None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:38.054987 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 343/708] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 31 01:04:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:04:38.060747 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89178c99-52e3-43fa-bb32-1dbb67f92299 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:38.078223 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c868e5-8dfe-4789-a2a5-0650a205daf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:38.079501 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c868e5-8dfe-4789-a2a5-0650a205daf1 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:38.079928 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c868e5-8dfe-4789-a2a5-0650a205daf1 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:38.080347 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c868e5-8dfe-4789-a2a5-0650a205daf1 None admin] PATH_INFO: `/v3/projects` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:38.081232 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c868e5-8dfe-4789-a2a5-0650a205daf1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:04:38.081703 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c868e5-8dfe-4789-a2a5-0650a205daf1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:38.083300 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c868e5-8dfe-4789-a2a5-0650a205daf1 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:38.096470 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 366/709] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Aug 31 01:04:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:04:38.102525 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c868e5-8dfe-4789-a2a5-0650a205daf1 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:38.119965 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:38.217750 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c868e5-8dfe-4789-a2a5-0650a205daf1 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:38.252764 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89178c99-52e3-43fa-bb32-1dbb67f92299 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:38.259679 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f0b6c3-7b5e-4b8b-8a69-0234781a8067 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:38.260543 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f0b6c3-7b5e-4b8b-8a69-0234781a8067 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:38.261023 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f0b6c3-7b5e-4b8b-8a69-0234781a8067 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:38.261443 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f0b6c3-7b5e-4b8b-8a69-0234781a8067 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:38.262335 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f0b6c3-7b5e-4b8b-8a69-0234781a8067 None admin] RBAC: Adding query filter params (role.id=d009411496b8452b88e77ffb8db015a7, scope.project.id=60b2abc15c2849ba8a1ed92117cc981b, user.id=8744c9de803048f583d0317d01f6a540) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:04:38.263230 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f0b6c3-7b5e-4b8b-8a69-0234781a8067 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:38.265002 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f0b6c3-7b5e-4b8b-8a69-0234781a8067 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:38.272638 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 367/710] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Aug 31 01:04:38 2022] GET /identity/v3/role_assignments?user.id=8744c9de803048f583d0317d01f6a540&scope.project.id=60b2abc15c2849ba8a1ed92117cc981b&role.id=d009411496b8452b88e77ffb8db015a7 => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:04:38.281422 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f0b6c3-7b5e-4b8b-8a69-0234781a8067 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:38.329998 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e5e5e0-969b-48c0-aa1f-11c65ccc3f47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:38.330689 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e5e5e0-969b-48c0-aa1f-11c65ccc3f47 None admin] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:38.330973 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e5e5e0-969b-48c0-aa1f-11c65ccc3f47 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:38.331245 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e5e5e0-969b-48c0-aa1f-11c65ccc3f47 None admin] PATH_INFO: `/v3/users` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:38.331912 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e5e5e0-969b-48c0-aa1f-11c65ccc3f47 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:38.336052 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e5e5e0-969b-48c0-aa1f-11c65ccc3f47 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:38.429660 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:38.437585 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f0b6c3-7b5e-4b8b-8a69-0234781a8067 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:38.456954 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 344/711] 10.222.0.20 () {60 vars in 1132 bytes} [Wed Aug 31 01:04:38 2022] POST /identity/v3/users => generated 254 bytes in 392 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:04:38.476970 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1356d182-2574-4694-a296-5aed20792477 None None] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:38.477304 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1356d182-2574-4694-a296-5aed20792477 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:38.477563 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1356d182-2574-4694-a296-5aed20792477 None None] PATH_INFO: `/` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:38.479038 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 345/712] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:04:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 01:04:38.486169 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0abd55d-cb72-4df7-9f56-92d1a84ebcbf None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:38.486781 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0abd55d-cb72-4df7-9f56-92d1a84ebcbf None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:38.487294 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0abd55d-cb72-4df7-9f56-92d1a84ebcbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:38.504267 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f39c382-f652-446b-af53-f5a1c66bc25c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:38.506202 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f39c382-f652-446b-af53-f5a1c66bc25c None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:38.506485 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f39c382-f652-446b-af53-f5a1c66bc25c None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:38.506761 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f39c382-f652-446b-af53-f5a1c66bc25c None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:38.512210 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f39c382-f652-446b-af53-f5a1c66bc25c None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:38.513809 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f39c382-f652-446b-af53-f5a1c66bc25c None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:38.533828 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-a0abd55d-cb72-4df7-9f56-92d1a84ebcbf None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:04:38.535002 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-a0abd55d-cb72-4df7-9f56-92d1a84ebcbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:04:38.541216 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application [None req-1f39c382-f652-446b-af53-f5a1c66bc25c None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 31 01:04:38.541216 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:04:38.541216 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 01:04:38.541216 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 01:04:38.541216 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 01:04:38.541216 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 01:04:38.541216 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 01:04:38.541216 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 01:04:38.541216 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 31 01:04:38.541216 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Aug 31 01:04:38.541216 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 01:04:38.541216 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Aug 31 01:04:38.541216 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:04:38.541216 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:04:38.541216 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:04:38.541216 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:04:38.541216 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:04:38.542320 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:04:38.542320 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:04:38.542320 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:04:38.542320 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:04:38.542320 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:04:38.542320 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:04:38.542320 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 01:04:38.542320 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 01:04:38.542320 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 01:04:38.542320 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 01:04:38.542320 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:04:38.542320 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:04:38.542320 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:04:38.542320 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:04:38.542320 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:04:38.542320 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:04:38.543261 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:04:38.543261 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application with Lock( Aug 31 01:04:38.543261 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:04:38.543261 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:04:38.543261 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:04:38.543261 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:04:38.543261 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:04:38.543261 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:04:38.543261 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:04:38.543261 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:04:38.543261 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 01:04:38.543261 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 01:04:38.543261 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 31 01:04:38.543261 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Aug 31 01:04:38.550020 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 368/713] 10.222.0.20 () {56 vars in 1129 bytes} [Wed Aug 31 01:04:38 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:04:38.565107 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f39c382-f652-446b-af53-f5a1c66bc25c None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:38.630674 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:38.658293 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 346/714] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:04:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 01:04:38.704285 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f39c382-f652-446b-af53-f5a1c66bc25c None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:38.738810 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7c51d7-1085-48ed-bb2e-b2c15014ed64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:38.739769 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7c51d7-1085-48ed-bb2e-b2c15014ed64 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:38.740257 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7c51d7-1085-48ed-bb2e-b2c15014ed64 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:38.740755 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7c51d7-1085-48ed-bb2e-b2c15014ed64 None admin] PATH_INFO: `/v3/services` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:38.742192 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7c51d7-1085-48ed-bb2e-b2c15014ed64 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:04:38.742722 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7c51d7-1085-48ed-bb2e-b2c15014ed64 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:38.744326 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7c51d7-1085-48ed-bb2e-b2c15014ed64 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:38.749592 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 369/715] 10.222.0.20 () {56 vars in 1124 bytes} [Wed Aug 31 01:04:38 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:04:38.755040 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0abd55d-cb72-4df7-9f56-92d1a84ebcbf None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:38.863867 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0abd55d-cb72-4df7-9f56-92d1a84ebcbf None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:38.909700 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ee3093-39cc-423b-a2d8-694637483177 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:38.910330 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ee3093-39cc-423b-a2d8-694637483177 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:38.910613 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ee3093-39cc-423b-a2d8-694637483177 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:38.910888 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ee3093-39cc-423b-a2d8-694637483177 None admin] PATH_INFO: `/v3/services` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:38.911545 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ee3093-39cc-423b-a2d8-694637483177 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:04:38.911851 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ee3093-39cc-423b-a2d8-694637483177 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:38.913327 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ee3093-39cc-423b-a2d8-694637483177 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:38.920759 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 347/716] 10.222.0.20 () {56 vars in 1124 bytes} [Wed Aug 31 01:04:38 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 01:04:38.959442 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7c51d7-1085-48ed-bb2e-b2c15014ed64 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:38.966810 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:39.081128 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7c51d7-1085-48ed-bb2e-b2c15014ed64 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:39.122016 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d08b85-14c9-484d-bef4-c03309280222 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:39.122582 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d08b85-14c9-484d-bef4-c03309280222 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:39.122867 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d08b85-14c9-484d-bef4-c03309280222 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:39.123139 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d08b85-14c9-484d-bef4-c03309280222 None admin] PATH_INFO: `/v3/services/network` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:39.123803 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d08b85-14c9-484d-bef4-c03309280222 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:39.125317 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d08b85-14c9-484d-bef4-c03309280222 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:39.135981 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application [None req-21d08b85-14c9-484d-bef4-c03309280222 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 31 01:04:39.135981 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:04:39.135981 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 01:04:39.135981 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 01:04:39.135981 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 01:04:39.135981 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 01:04:39.135981 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 01:04:39.135981 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 01:04:39.135981 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 31 01:04:39.135981 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Aug 31 01:04:39.135981 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 01:04:39.135981 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Aug 31 01:04:39.135981 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:04:39.135981 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:04:39.135981 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:04:39.135981 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:04:39.135981 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:04:39.137036 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:04:39.137036 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:04:39.137036 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:04:39.137036 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:04:39.137036 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:04:39.137036 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:04:39.137036 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 01:04:39.137036 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 01:04:39.137036 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 01:04:39.137036 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 01:04:39.137036 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:04:39.137036 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:04:39.137036 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:04:39.137036 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:04:39.137036 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:04:39.137036 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:04:39.138489 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:04:39.138489 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application with Lock( Aug 31 01:04:39.138489 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:04:39.138489 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:04:39.138489 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:04:39.138489 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:04:39.138489 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:04:39.138489 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:04:39.138489 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:04:39.138489 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:04:39.138489 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 01:04:39.138489 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 01:04:39.138489 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 31 01:04:39.138489 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Aug 31 01:04:39.138489 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 370/717] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Aug 31 01:04:38 2022] GET /identity/v3/services/network => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:04:39.143410 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ee3093-39cc-423b-a2d8-694637483177 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:39.211412 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:39.289771 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ee3093-39cc-423b-a2d8-694637483177 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:39.337985 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbc78ca-c70a-46da-b08b-24ae3e15f51f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:39.338602 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbc78ca-c70a-46da-b08b-24ae3e15f51f None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:39.338887 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbc78ca-c70a-46da-b08b-24ae3e15f51f None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:39.339167 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbc78ca-c70a-46da-b08b-24ae3e15f51f None admin] PATH_INFO: `/v3/services` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:39.339799 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbc78ca-c70a-46da-b08b-24ae3e15f51f None admin] RBAC: Adding query filter params (name=network) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:04:39.340103 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbc78ca-c70a-46da-b08b-24ae3e15f51f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:39.341512 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbc78ca-c70a-46da-b08b-24ae3e15f51f None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:39.348674 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 348/718] 10.222.0.20 () {56 vars in 1112 bytes} [Wed Aug 31 01:04:39 2022] GET /identity/v3/services?name=network => generated 124 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:04:39.354983 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d08b85-14c9-484d-bef4-c03309280222 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:39.455755 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:39.497351 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d08b85-14c9-484d-bef4-c03309280222 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:39.540277 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9239d3cd-ec5a-4a90-82e0-39b8392839dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:39.540941 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9239d3cd-ec5a-4a90-82e0-39b8392839dd None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:39.541230 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9239d3cd-ec5a-4a90-82e0-39b8392839dd None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:39.541509 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9239d3cd-ec5a-4a90-82e0-39b8392839dd None admin] PATH_INFO: `/v3/services` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:39.542144 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9239d3cd-ec5a-4a90-82e0-39b8392839dd None admin] RBAC: Adding query filter params (type=network) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:04:39.542448 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9239d3cd-ec5a-4a90-82e0-39b8392839dd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:39.543816 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9239d3cd-ec5a-4a90-82e0-39b8392839dd None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:39.553847 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 371/719] 10.222.0.20 () {56 vars in 1112 bytes} [Wed Aug 31 01:04:39 2022] GET /identity/v3/services?type=network => generated 124 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:04:39.870233 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057598cf-715d-4bd7-bb75-72f4ef15e856 None None] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:39.870436 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057598cf-715d-4bd7-bb75-72f4ef15e856 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:39.870704 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057598cf-715d-4bd7-bb75-72f4ef15e856 None None] PATH_INFO: `/` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:39.872312 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 349/720] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:04:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:04:39.877587 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93928e38-a3cd-46ba-b227-f5013dcff521 None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:39.877874 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93928e38-a3cd-46ba-b227-f5013dcff521 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:39.878196 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93928e38-a3cd-46ba-b227-f5013dcff521 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:39.906421 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-93928e38-a3cd-46ba-b227-f5013dcff521 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:04:39.907605 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-93928e38-a3cd-46ba-b227-f5013dcff521 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:04:39.984841 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 372/721] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:04:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 01:04:40.307229 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057598cf-715d-4bd7-bb75-72f4ef15e856 None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:40.436375 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057598cf-715d-4bd7-bb75-72f4ef15e856 None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:40.438712 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:40.471351 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a424c0eb-506e-4e5d-bc3a-627c4cb89b5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:40.472041 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a424c0eb-506e-4e5d-bc3a-627c4cb89b5d None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:40.472869 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a424c0eb-506e-4e5d-bc3a-627c4cb89b5d None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:40.473655 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a424c0eb-506e-4e5d-bc3a-627c4cb89b5d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:40.484516 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a424c0eb-506e-4e5d-bc3a-627c4cb89b5d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 01:04:40.489312 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application [None req-a424c0eb-506e-4e5d-bc3a-627c4cb89b5d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 01:04:40.489312 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:04:40.489312 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:04:40.489312 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:04:40.489312 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:04:40.489312 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:04:40.489312 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:04:40.489312 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:04:40.489312 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:04:40.489312 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:04:40.489312 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:04:40.489312 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:04:40.489312 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 01:04:40.489312 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 01:04:40.489312 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 01:04:40.489312 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 01:04:40.490232 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 01:04:40.490232 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 01:04:40.490232 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 01:04:40.490232 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 01:04:40.490232 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:04:40.490232 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:04:40.490232 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:04:40.490232 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:04:40.490232 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:04:40.490232 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:04:40.490232 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:04:40.490232 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application with Lock( Aug 31 01:04:40.490232 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:04:40.490232 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:04:40.490232 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:04:40.490232 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:04:40.491070 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:04:40.491070 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:04:40.491070 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:04:40.491070 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:04:40.491070 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 01:04:40.491070 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 01:04:40.491070 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 01:04:40.491070 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 01:04:40.491070 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 01:04:40.491070 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 01:04:40.491070 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 01:04:40.491070 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Aug 31 01:04:40.491070 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 350/722] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Aug 31 01:04:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:04:40.497943 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93928e38-a3cd-46ba-b227-f5013dcff521 None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:40.574777 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:40.614644 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93928e38-a3cd-46ba-b227-f5013dcff521 None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:40.650150 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912d3469-1d09-42eb-be34-6aca8b254a76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:40.650757 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912d3469-1d09-42eb-be34-6aca8b254a76 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:40.651033 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912d3469-1d09-42eb-be34-6aca8b254a76 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:40.651304 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912d3469-1d09-42eb-be34-6aca8b254a76 None admin] PATH_INFO: `/v3/roles` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:40.651954 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912d3469-1d09-42eb-be34-6aca8b254a76 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:04:40.652305 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912d3469-1d09-42eb-be34-6aca8b254a76 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:40.653872 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912d3469-1d09-42eb-be34-6aca8b254a76 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:40.662455 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 373/723] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Aug 31 01:04:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:04:40.674119 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a424c0eb-506e-4e5d-bc3a-627c4cb89b5d None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:40.692422 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ceadbe-c49d-40a0-a4ed-13d802aa84f1 None None] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:40.692703 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ceadbe-c49d-40a0-a4ed-13d802aa84f1 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:40.693001 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ceadbe-c49d-40a0-a4ed-13d802aa84f1 None None] PATH_INFO: `/` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:40.694516 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 374/724] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:04:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:04:40.705893 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f20302-b754-4618-82f9-42a5d9bd8c80 None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:40.706172 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f20302-b754-4618-82f9-42a5d9bd8c80 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:40.706445 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f20302-b754-4618-82f9-42a5d9bd8c80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:40.739693 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-33f20302-b754-4618-82f9-42a5d9bd8c80 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:04:40.740910 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-33f20302-b754-4618-82f9-42a5d9bd8c80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:04:40.833814 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a424c0eb-506e-4e5d-bc3a-627c4cb89b5d None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:40.870532 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a66d50d-8944-48b8-96c8-3825640baaef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:40.871165 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a66d50d-8944-48b8-96c8-3825640baaef None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:40.871498 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a66d50d-8944-48b8-96c8-3825640baaef None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:40.871772 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a66d50d-8944-48b8-96c8-3825640baaef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:40.884641 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a66d50d-8944-48b8-96c8-3825640baaef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:40.886304 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a66d50d-8944-48b8-96c8-3825640baaef None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:40.914446 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 375/725] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:04:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 01:04:40.916277 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application [None req-7a66d50d-8944-48b8-96c8-3825640baaef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 01:04:40.916277 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:04:40.916277 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:04:40.916277 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:04:40.916277 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:04:40.916277 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:04:40.916277 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:04:40.916277 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:04:40.916277 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:04:40.916277 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:04:40.916277 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:04:40.916277 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:04:40.916277 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 01:04:40.916277 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 01:04:40.916277 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 01:04:40.916277 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 01:04:40.917310 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:04:40.917310 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:04:40.917310 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:04:40.917310 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:04:40.917310 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:04:40.917310 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:04:40.917310 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:04:40.917310 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application with Lock( Aug 31 01:04:40.917310 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:04:40.917310 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:04:40.917310 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:04:40.917310 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:04:40.917310 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:04:40.917310 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:04:40.917310 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:04:40.917310 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:04:40.917310 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 01:04:40.918220 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 01:04:40.918220 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 01:04:40.918220 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Aug 31 01:04:40.918220 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 351/726] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 01:04:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:04:40.924418 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f20302-b754-4618-82f9-42a5d9bd8c80 None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:41.001328 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:41.060001 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f20302-b754-4618-82f9-42a5d9bd8c80 None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:41.110801 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5eba4ec-f89e-423f-a6c1-c29999881c46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:41.111332 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5eba4ec-f89e-423f-a6c1-c29999881c46 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:41.111612 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5eba4ec-f89e-423f-a6c1-c29999881c46 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:41.111885 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5eba4ec-f89e-423f-a6c1-c29999881c46 None admin] PATH_INFO: `/v3/domains` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:41.112521 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5eba4ec-f89e-423f-a6c1-c29999881c46 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:04:41.112852 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5eba4ec-f89e-423f-a6c1-c29999881c46 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:41.114222 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5eba4ec-f89e-423f-a6c1-c29999881c46 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:41.134175 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 376/727] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 31 01:04:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:04:41.140025 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a66d50d-8944-48b8-96c8-3825640baaef None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:41.172246 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:41.245051 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5eba4ec-f89e-423f-a6c1-c29999881c46 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:41.316625 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a66d50d-8944-48b8-96c8-3825640baaef None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:41.356680 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d5ca25-0434-43b0-85fe-dfac0d58464c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:41.357438 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d5ca25-0434-43b0-85fe-dfac0d58464c None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:41.357833 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d5ca25-0434-43b0-85fe-dfac0d58464c None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:41.358112 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d5ca25-0434-43b0-85fe-dfac0d58464c None admin] PATH_INFO: `/v3/users/swift` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:41.360607 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5eba4ec-f89e-423f-a6c1-c29999881c46 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:41.366258 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d5ca25-0434-43b0-85fe-dfac0d58464c None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:41.368293 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d5ca25-0434-43b0-85fe-dfac0d58464c None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:41.375293 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:41.381362 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application [None req-83d5ca25-0434-43b0-85fe-dfac0d58464c None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 31 01:04:41.381362 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:04:41.381362 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:04:41.381362 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:04:41.381362 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:04:41.381362 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:04:41.381362 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:04:41.381362 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:04:41.381362 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:04:41.381362 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:04:41.381362 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:04:41.381362 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:04:41.381362 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 01:04:41.381362 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 01:04:41.381362 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 01:04:41.381362 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 01:04:41.382913 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:04:41.382913 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:04:41.382913 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 01:04:41.382913 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 01:04:41.382913 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 01:04:41.382913 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 01:04:41.382913 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:04:41.382913 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:04:41.382913 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:04:41.382913 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:04:41.382913 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:04:41.382913 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application with Lock( Aug 31 01:04:41.382913 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:04:41.382913 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:04:41.382913 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:04:41.382913 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:04:41.382913 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:04:41.382913 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:04:41.383848 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:04:41.383848 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:04:41.383848 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 01:04:41.383848 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 01:04:41.383848 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 01:04:41.383848 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 01:04:41.383848 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 01:04:41.383848 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 01:04:41.383848 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 31 01:04:41.383848 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Aug 31 01:04:41.383848 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 352/728] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Aug 31 01:04:41 2022] GET /identity/v3/users/swift => generated 83 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:04:41.384980 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c73046b-6f35-4758-a8eb-7cabfe40c3e1 None None] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:41.384980 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c73046b-6f35-4758-a8eb-7cabfe40c3e1 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:41.384980 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c73046b-6f35-4758-a8eb-7cabfe40c3e1 None None] PATH_INFO: `/` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:41.394036 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 353/729] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:04:41 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:04:41.394036 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e7b450-a06c-42fd-86e9-cbb5c6674d75 None None] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:41.394036 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e7b450-a06c-42fd-86e9-cbb5c6674d75 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:41.394036 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e7b450-a06c-42fd-86e9-cbb5c6674d75 None None] PATH_INFO: `/` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:41.394036 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 354/730] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:04:41 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:04:41.394036 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e7b450-a06c-42fd-86e9-cbb5c6674d75 None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:41.400048 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbffe584-b686-41c0-9050-a934ad55c7d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:41.400687 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbffe584-b686-41c0-9050-a934ad55c7d4 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:41.401004 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbffe584-b686-41c0-9050-a934ad55c7d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:41.401279 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbffe584-b686-41c0-9050-a934ad55c7d4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:41.412327 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbffe584-b686-41c0-9050-a934ad55c7d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 01:04:41.418385 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application [None req-bbffe584-b686-41c0-9050-a934ad55c7d4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 01:04:41.418385 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:04:41.418385 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:04:41.418385 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:04:41.418385 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:04:41.418385 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:04:41.418385 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:04:41.418385 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:04:41.418385 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:04:41.418385 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:04:41.418385 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:04:41.418385 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:04:41.418385 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 01:04:41.418385 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 01:04:41.418385 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 01:04:41.418385 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 01:04:41.419376 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 01:04:41.419376 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 01:04:41.419376 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 01:04:41.419376 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 01:04:41.419376 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:04:41.419376 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:04:41.419376 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:04:41.419376 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:04:41.419376 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:04:41.419376 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:04:41.419376 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:04:41.419376 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application with Lock( Aug 31 01:04:41.419376 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:04:41.419376 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:04:41.419376 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:04:41.419376 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:04:41.420276 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:04:41.420276 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:04:41.420276 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:04:41.420276 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:04:41.420276 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 01:04:41.420276 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 01:04:41.420276 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 01:04:41.420276 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 01:04:41.420276 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 01:04:41.420276 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 01:04:41.420276 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 01:04:41.420276 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Aug 31 01:04:41.424886 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 377/731] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Aug 31 01:04:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:04:41.426016 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bf8b82-9c50-4443-90d2-3fd1af28f1dc None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:41.426306 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bf8b82-9c50-4443-90d2-3fd1af28f1dc None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:41.426588 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bf8b82-9c50-4443-90d2-3fd1af28f1dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:41.444342 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-84bf8b82-9c50-4443-90d2-3fd1af28f1dc None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:04:41.446122 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-84bf8b82-9c50-4443-90d2-3fd1af28f1dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:04:41.479304 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:41.503246 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e7b450-a06c-42fd-86e9-cbb5c6674d75 None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:41.524915 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 378/732] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:04:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 01:04:41.528748 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc29997-69cb-4d38-a05c-8049856c95ba None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:41.528748 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc29997-69cb-4d38-a05c-8049856c95ba None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:41.528748 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc29997-69cb-4d38-a05c-8049856c95ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:41.539834 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07ed323-5352-4576-a6dd-02ce224e754d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:41.540439 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07ed323-5352-4576-a6dd-02ce224e754d None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:41.540736 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07ed323-5352-4576-a6dd-02ce224e754d None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:41.541032 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07ed323-5352-4576-a6dd-02ce224e754d None admin] PATH_INFO: `/v3/users` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:41.541764 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07ed323-5352-4576-a6dd-02ce224e754d None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:04:41.542066 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07ed323-5352-4576-a6dd-02ce224e754d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:41.543486 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07ed323-5352-4576-a6dd-02ce224e754d None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:41.550169 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-bdc29997-69cb-4d38-a05c-8049856c95ba None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:04:41.551651 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-bdc29997-69cb-4d38-a05c-8049856c95ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:04:41.566510 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 355/733] 10.222.0.20 () {56 vars in 1135 bytes} [Wed Aug 31 01:04:41 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:04:41.567392 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07ed323-5352-4576-a6dd-02ce224e754d None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:41.645264 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 379/734] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:04:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 01:04:41.646054 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc29997-69cb-4d38-a05c-8049856c95ba None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:41.685456 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:41.704401 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07ed323-5352-4576-a6dd-02ce224e754d None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:41.764232 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fedd77c-a2d4-4348-b572-5a51b8119fb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:41.764909 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fedd77c-a2d4-4348-b572-5a51b8119fb9 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:41.765317 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fedd77c-a2d4-4348-b572-5a51b8119fb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:41.765588 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fedd77c-a2d4-4348-b572-5a51b8119fb9 None admin] PATH_INFO: `/v3/roles` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:41.766255 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fedd77c-a2d4-4348-b572-5a51b8119fb9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:04:41.766558 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fedd77c-a2d4-4348-b572-5a51b8119fb9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:41.768187 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fedd77c-a2d4-4348-b572-5a51b8119fb9 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:41.769604 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:41.778285 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 356/735] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Aug 31 01:04:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:04:41.783371 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fedd77c-a2d4-4348-b572-5a51b8119fb9 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:41.810944 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc29997-69cb-4d38-a05c-8049856c95ba None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:41.862294 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf542d7-6521-4cb9-ac42-f03db00ae850 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:41.863529 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf542d7-6521-4cb9-ac42-f03db00ae850 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:41.864258 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf542d7-6521-4cb9-ac42-f03db00ae850 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:41.864750 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf542d7-6521-4cb9-ac42-f03db00ae850 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:41.904075 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fedd77c-a2d4-4348-b572-5a51b8119fb9 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:41.940395 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847f6754-e157-48df-ae48-eb2fcd06f5b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:41.941264 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847f6754-e157-48df-ae48-eb2fcd06f5b8 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:41.941621 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847f6754-e157-48df-ae48-eb2fcd06f5b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:41.942050 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847f6754-e157-48df-ae48-eb2fcd06f5b8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:41.954110 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847f6754-e157-48df-ae48-eb2fcd06f5b8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:41.955590 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847f6754-e157-48df-ae48-eb2fcd06f5b8 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:41.958670 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf542d7-6521-4cb9-ac42-f03db00ae850 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:41.960617 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf542d7-6521-4cb9-ac42-f03db00ae850 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:41.970838 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application [None req-847f6754-e157-48df-ae48-eb2fcd06f5b8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 01:04:41.970838 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:04:41.970838 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:04:41.970838 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:04:41.970838 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:04:41.970838 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:04:41.970838 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:04:41.970838 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:04:41.970838 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:04:41.970838 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:04:41.970838 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:04:41.970838 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:04:41.970838 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 01:04:41.970838 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 01:04:41.970838 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 01:04:41.970838 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 01:04:41.971805 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:04:41.971805 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:04:41.971805 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:04:41.971805 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:04:41.971805 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:04:41.971805 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:04:41.971805 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:04:41.971805 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application with Lock( Aug 31 01:04:41.971805 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:04:41.971805 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:04:41.971805 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:04:41.971805 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:04:41.971805 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:04:41.971805 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:04:41.971805 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:04:41.971805 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:04:41.971805 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 01:04:41.972822 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 01:04:41.972822 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 01:04:41.972822 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Aug 31 01:04:41.972822 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 357/736] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 01:04:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:04:41.972822 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847f6754-e157-48df-ae48-eb2fcd06f5b8 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:41.997805 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:42.020805 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:42.062907 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 380/737] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Aug 31 01:04:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 01:04:42.064194 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14189c84-01d2-4276-98b7-4c865438e65f None None] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:42.064484 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14189c84-01d2-4276-98b7-4c865438e65f None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:42.064810 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14189c84-01d2-4276-98b7-4c865438e65f None None] PATH_INFO: `/` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:42.069084 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 381/738] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:04:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:04:42.069084 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14189c84-01d2-4276-98b7-4c865438e65f None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:42.075511 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847f6754-e157-48df-ae48-eb2fcd06f5b8 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:42.108968 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167a48db-8137-484b-9673-a2e3e4c7a80e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:42.109582 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167a48db-8137-484b-9673-a2e3e4c7a80e None admin] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:42.109878 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167a48db-8137-484b-9673-a2e3e4c7a80e None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:42.110142 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167a48db-8137-484b-9673-a2e3e4c7a80e None admin] PATH_INFO: `/v3/services` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:42.110789 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167a48db-8137-484b-9673-a2e3e4c7a80e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:42.112149 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167a48db-8137-484b-9673-a2e3e4c7a80e None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:42.166966 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14189c84-01d2-4276-98b7-4c865438e65f None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:42.198090 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389b9a13-bac9-4e0a-97b0-0d4ccdf9eeef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:42.198705 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389b9a13-bac9-4e0a-97b0-0d4ccdf9eeef None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:42.198969 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389b9a13-bac9-4e0a-97b0-0d4ccdf9eeef None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:42.199239 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389b9a13-bac9-4e0a-97b0-0d4ccdf9eeef None admin] PATH_INFO: `/v3/roles/service` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:42.201280 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:42.209919 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389b9a13-bac9-4e0a-97b0-0d4ccdf9eeef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 01:04:42.215237 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application [None req-389b9a13-bac9-4e0a-97b0-0d4ccdf9eeef None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 01:04:42.215237 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:04:42.215237 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:04:42.215237 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:04:42.215237 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:04:42.215237 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:04:42.215237 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:04:42.215237 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:04:42.215237 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:04:42.215237 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:04:42.215237 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:04:42.215237 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:04:42.215237 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 01:04:42.215237 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 01:04:42.215237 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 01:04:42.215237 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 01:04:42.216344 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 01:04:42.216344 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 01:04:42.216344 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 01:04:42.216344 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 01:04:42.216344 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:04:42.216344 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:04:42.216344 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:04:42.216344 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:04:42.216344 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:04:42.216344 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:04:42.216344 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:04:42.216344 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application with Lock( Aug 31 01:04:42.216344 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:04:42.216344 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:04:42.216344 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:04:42.216344 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:04:42.217231 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:04:42.217231 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:04:42.217231 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:04:42.217231 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:04:42.217231 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 01:04:42.217231 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 01:04:42.217231 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 01:04:42.217231 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 01:04:42.217231 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 01:04:42.217231 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 01:04:42.217231 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 01:04:42.217231 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Aug 31 01:04:42.217231 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 382/739] 10.222.0.20 () {56 vars in 1102 bytes} [Wed Aug 31 01:04:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:04:42.217231 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389b9a13-bac9-4e0a-97b0-0d4ccdf9eeef None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:42.224763 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 358/740] 10.222.0.20 () {60 vars in 1141 bytes} [Wed Aug 31 01:04:41 2022] POST /identity/v3/services => generated 251 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:04:42.225198 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167a48db-8137-484b-9673-a2e3e4c7a80e None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:42.341232 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389b9a13-bac9-4e0a-97b0-0d4ccdf9eeef None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:42.352132 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167a48db-8137-484b-9673-a2e3e4c7a80e None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:42.357470 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:42.380373 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac274a8-adf2-4d93-a8dc-b5de68575527 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:42.381034 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac274a8-adf2-4d93-a8dc-b5de68575527 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:42.381319 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac274a8-adf2-4d93-a8dc-b5de68575527 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:42.381593 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac274a8-adf2-4d93-a8dc-b5de68575527 None admin] PATH_INFO: `/v3/domains` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:42.382758 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac274a8-adf2-4d93-a8dc-b5de68575527 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:04:42.383060 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac274a8-adf2-4d93-a8dc-b5de68575527 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:42.384441 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac274a8-adf2-4d93-a8dc-b5de68575527 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:42.396207 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b39274-4773-4e1c-9a50-3d68beb74a32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:42.397296 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b39274-4773-4e1c-9a50-3d68beb74a32 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:42.397423 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 383/741] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 31 01:04:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:04:42.397503 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b39274-4773-4e1c-9a50-3d68beb74a32 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:42.397503 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b39274-4773-4e1c-9a50-3d68beb74a32 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:42.398828 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a755b196-0331-4cce-8ecc-0507b7f187e8 None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:42.399122 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a755b196-0331-4cce-8ecc-0507b7f187e8 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:42.399397 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a755b196-0331-4cce-8ecc-0507b7f187e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:42.413111 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b39274-4773-4e1c-9a50-3d68beb74a32 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:42.414619 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b39274-4773-4e1c-9a50-3d68beb74a32 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:42.419040 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-a755b196-0331-4cce-8ecc-0507b7f187e8 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:04:42.420205 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-a755b196-0331-4cce-8ecc-0507b7f187e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:04:42.430141 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application [None req-a4b39274-4773-4e1c-9a50-3d68beb74a32 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 01:04:42.430141 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:04:42.430141 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:04:42.430141 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:04:42.430141 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:04:42.430141 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:04:42.430141 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:04:42.430141 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:04:42.430141 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:04:42.430141 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:04:42.430141 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:04:42.430141 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:04:42.430141 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 01:04:42.430141 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 01:04:42.430141 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 01:04:42.430141 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 01:04:42.431324 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:04:42.431324 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:04:42.431324 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:04:42.431324 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:04:42.431324 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:04:42.431324 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:04:42.431324 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:04:42.431324 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application with Lock( Aug 31 01:04:42.431324 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:04:42.431324 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:04:42.431324 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:04:42.431324 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:04:42.431324 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:04:42.431324 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:04:42.431324 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:04:42.431324 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:04:42.431324 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 01:04:42.432464 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 01:04:42.432464 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 01:04:42.432464 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Aug 31 01:04:42.432464 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 359/742] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 01:04:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:04:42.432464 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b39274-4773-4e1c-9a50-3d68beb74a32 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:42.478617 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:42.497082 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 384/743] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:04:42 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 01:04:42.497923 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a755b196-0331-4cce-8ecc-0507b7f187e8 None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:42.556074 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b39274-4773-4e1c-9a50-3d68beb74a32 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:42.592414 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b14103-ffe9-4763-a016-fd6c32357d43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:42.593057 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b14103-ffe9-4763-a016-fd6c32357d43 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:42.593343 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b14103-ffe9-4763-a016-fd6c32357d43 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:42.593625 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b14103-ffe9-4763-a016-fd6c32357d43 None admin] PATH_INFO: `/v3/roles` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:42.594405 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b14103-ffe9-4763-a016-fd6c32357d43 None admin] RBAC: Adding query filter params (name=service) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:04:42.594718 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b14103-ffe9-4763-a016-fd6c32357d43 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:42.596572 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b14103-ffe9-4763-a016-fd6c32357d43 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:42.611882 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 360/744] 10.222.0.20 () {56 vars in 1103 bytes} [Wed Aug 31 01:04:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:04:42.612823 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b14103-ffe9-4763-a016-fd6c32357d43 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:42.627870 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a755b196-0331-4cce-8ecc-0507b7f187e8 None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:42.637062 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:42.668304 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4929653-3ea9-4e77-8c70-1df5231fa12d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:42.669013 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4929653-3ea9-4e77-8c70-1df5231fa12d None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:42.669332 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4929653-3ea9-4e77-8c70-1df5231fa12d None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:42.669555 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4929653-3ea9-4e77-8c70-1df5231fa12d None admin] PATH_INFO: `/v3/users/nova` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:42.677432 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4929653-3ea9-4e77-8c70-1df5231fa12d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:42.679074 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4929653-3ea9-4e77-8c70-1df5231fa12d None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:42.696845 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application [None req-b4929653-3ea9-4e77-8c70-1df5231fa12d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 31 01:04:42.696845 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:04:42.696845 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:04:42.696845 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:04:42.696845 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:04:42.696845 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:04:42.696845 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:04:42.696845 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:04:42.696845 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:04:42.696845 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:04:42.696845 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:04:42.696845 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:04:42.696845 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 01:04:42.696845 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 01:04:42.696845 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 01:04:42.696845 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 01:04:42.700062 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:04:42.700062 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:04:42.700062 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 01:04:42.700062 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 01:04:42.700062 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 01:04:42.700062 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 01:04:42.700062 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:04:42.700062 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:04:42.700062 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:04:42.700062 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:04:42.700062 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:04:42.700062 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application with Lock( Aug 31 01:04:42.700062 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:04:42.700062 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:04:42.700062 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:04:42.700062 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:04:42.700062 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:04:42.700062 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:04:42.701008 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:04:42.701008 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:04:42.701008 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 01:04:42.701008 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 01:04:42.701008 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 01:04:42.701008 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 01:04:42.701008 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 01:04:42.701008 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 01:04:42.701008 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 31 01:04:42.701008 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Aug 31 01:04:42.701008 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 385/745] 10.222.0.20 () {56 vars in 1093 bytes} [Wed Aug 31 01:04:42 2022] GET /identity/v3/users/nova => generated 82 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:04:42.701008 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4929653-3ea9-4e77-8c70-1df5231fa12d None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:42.745493 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:42.747700 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b14103-ffe9-4763-a016-fd6c32357d43 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:42.788683 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2e19cd-5bb1-4f6c-b869-ac5ed146a4f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:42.789321 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2e19cd-5bb1-4f6c-b869-ac5ed146a4f2 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:42.789609 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2e19cd-5bb1-4f6c-b869-ac5ed146a4f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:42.789891 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2e19cd-5bb1-4f6c-b869-ac5ed146a4f2 None admin] PATH_INFO: `/v3/domains` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:42.790541 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2e19cd-5bb1-4f6c-b869-ac5ed146a4f2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:04:42.790846 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2e19cd-5bb1-4f6c-b869-ac5ed146a4f2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:42.792283 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2e19cd-5bb1-4f6c-b869-ac5ed146a4f2 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:42.805595 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 361/746] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 31 01:04:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:04:42.806418 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4929653-3ea9-4e77-8c70-1df5231fa12d None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:42.806527 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2e19cd-5bb1-4f6c-b869-ac5ed146a4f2 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:42.843250 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f46285d-57c8-4b35-9a85-376c94494304 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:42.843879 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f46285d-57c8-4b35-9a85-376c94494304 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:42.844167 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f46285d-57c8-4b35-9a85-376c94494304 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:42.844446 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f46285d-57c8-4b35-9a85-376c94494304 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:42.855865 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f46285d-57c8-4b35-9a85-376c94494304 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:42.857452 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f46285d-57c8-4b35-9a85-376c94494304 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:42.871670 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application [None req-0f46285d-57c8-4b35-9a85-376c94494304 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 01:04:42.871670 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:04:42.871670 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:04:42.871670 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:04:42.871670 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:04:42.871670 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:04:42.871670 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:04:42.871670 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:04:42.871670 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:04:42.871670 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:04:42.871670 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:04:42.871670 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:04:42.871670 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 01:04:42.871670 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 01:04:42.871670 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 01:04:42.871670 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 01:04:42.872872 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:04:42.872872 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:04:42.872872 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:04:42.872872 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:04:42.872872 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:04:42.872872 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:04:42.872872 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:04:42.872872 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application with Lock( Aug 31 01:04:42.872872 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:04:42.872872 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:04:42.872872 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:04:42.872872 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:04:42.872872 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:04:42.872872 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:04:42.872872 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:04:42.872872 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:04:42.872872 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 01:04:42.873803 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 01:04:42.873803 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 01:04:42.873803 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Aug 31 01:04:42.873803 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 386/747] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 01:04:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:04:42.874028 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f46285d-57c8-4b35-9a85-376c94494304 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:42.961986 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:42.985016 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2e19cd-5bb1-4f6c-b869-ac5ed146a4f2 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:43.013326 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:43.019523 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f46285d-57c8-4b35-9a85-376c94494304 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:43.021305 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07bc4ef-0bad-4202-8eae-54726ef7ac04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:43.022188 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07bc4ef-0bad-4202-8eae-54726ef7ac04 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:43.022528 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07bc4ef-0bad-4202-8eae-54726ef7ac04 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:43.023325 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07bc4ef-0bad-4202-8eae-54726ef7ac04 None admin] PATH_INFO: `/v3/users` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:43.024045 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07bc4ef-0bad-4202-8eae-54726ef7ac04 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:04:43.024355 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07bc4ef-0bad-4202-8eae-54726ef7ac04 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:43.025823 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07bc4ef-0bad-4202-8eae-54726ef7ac04 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:43.043202 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 362/748] 10.222.0.20 () {56 vars in 1133 bytes} [Wed Aug 31 01:04:42 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:04:43.044275 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07bc4ef-0bad-4202-8eae-54726ef7ac04 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:43.060034 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc07719a-52f0-43ff-8e62-d2c8b2b1d351 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:43.060748 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc07719a-52f0-43ff-8e62-d2c8b2b1d351 None admin] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:43.061099 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc07719a-52f0-43ff-8e62-d2c8b2b1d351 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:43.061388 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc07719a-52f0-43ff-8e62-d2c8b2b1d351 None admin] PATH_INFO: `/v3/services` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:43.062031 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc07719a-52f0-43ff-8e62-d2c8b2b1d351 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:43.063395 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc07719a-52f0-43ff-8e62-d2c8b2b1d351 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:43.149256 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07bc4ef-0bad-4202-8eae-54726ef7ac04 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:43.150768 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 387/749] 10.222.0.20 () {60 vars in 1141 bytes} [Wed Aug 31 01:04:42 2022] POST /identity/v3/services => generated 240 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:04:43.151823 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc07719a-52f0-43ff-8e62-d2c8b2b1d351 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:43.200676 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:43.205465 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1316f023-1d81-4f62-865b-1f34c8ab83ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:43.206128 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1316f023-1d81-4f62-865b-1f34c8ab83ea None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:43.206411 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1316f023-1d81-4f62-865b-1f34c8ab83ea None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:43.206686 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1316f023-1d81-4f62-865b-1f34c8ab83ea None admin] PATH_INFO: `/v3/projects/service` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:43.215708 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1316f023-1d81-4f62-865b-1f34c8ab83ea None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:43.217439 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1316f023-1d81-4f62-865b-1f34c8ab83ea None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:43.230754 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application [None req-1316f023-1d81-4f62-865b-1f34c8ab83ea None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 01:04:43.230754 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:04:43.230754 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:04:43.230754 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:04:43.230754 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:04:43.230754 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:04:43.230754 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:04:43.230754 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:04:43.230754 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:04:43.230754 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:04:43.230754 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:04:43.230754 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:04:43.230754 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 01:04:43.230754 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 01:04:43.230754 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 01:04:43.231731 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 01:04:43.231731 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:04:43.231731 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:04:43.231731 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:04:43.231731 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:04:43.231731 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:04:43.231731 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:04:43.231731 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:04:43.231731 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application with Lock( Aug 31 01:04:43.231731 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:04:43.231731 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:04:43.231731 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:04:43.231731 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:04:43.231731 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:04:43.231731 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:04:43.231731 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:04:43.231731 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:04:43.232615 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 01:04:43.232615 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 01:04:43.232615 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 01:04:43.232615 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 01:04:43.232615 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 01:04:43.232615 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 01:04:43.232615 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 01:04:43.232615 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Aug 31 01:04:43.232615 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 363/750] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Aug 31 01:04:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:04:43.233628 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1316f023-1d81-4f62-865b-1f34c8ab83ea None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:43.302745 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc07719a-52f0-43ff-8e62-d2c8b2b1d351 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:43.353491 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4be74f8-9ea0-46b8-b797-5418110b8316 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:43.354507 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4be74f8-9ea0-46b8-b797-5418110b8316 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:43.354619 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4be74f8-9ea0-46b8-b797-5418110b8316 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:43.355202 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4be74f8-9ea0-46b8-b797-5418110b8316 None admin] PATH_INFO: `/v3/domains` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:43.356939 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4be74f8-9ea0-46b8-b797-5418110b8316 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:04:43.356939 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4be74f8-9ea0-46b8-b797-5418110b8316 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:43.357217 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4be74f8-9ea0-46b8-b797-5418110b8316 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:43.376942 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 388/751] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 31 01:04:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:04:43.376942 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4be74f8-9ea0-46b8-b797-5418110b8316 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:43.383566 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1316f023-1d81-4f62-865b-1f34c8ab83ea None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:43.420789 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b6f424-e816-407b-a9ab-1799ced920d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:43.421749 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b6f424-e816-407b-a9ab-1799ced920d4 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:43.422258 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b6f424-e816-407b-a9ab-1799ced920d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:43.422715 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b6f424-e816-407b-a9ab-1799ced920d4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:43.425127 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:43.434608 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b6f424-e816-407b-a9ab-1799ced920d4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:43.436355 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b6f424-e816-407b-a9ab-1799ced920d4 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:43.453761 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application [None req-b9b6f424-e816-407b-a9ab-1799ced920d4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 01:04:43.453761 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:04:43.453761 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:04:43.453761 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:04:43.453761 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:04:43.453761 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:04:43.453761 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:04:43.453761 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:04:43.453761 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:04:43.453761 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:04:43.453761 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:04:43.453761 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:04:43.453761 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 01:04:43.453761 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 01:04:43.453761 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 01:04:43.453761 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 01:04:43.454751 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:04:43.454751 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:04:43.454751 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:04:43.454751 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:04:43.454751 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:04:43.454751 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:04:43.454751 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:04:43.454751 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application with Lock( Aug 31 01:04:43.454751 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:04:43.454751 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:04:43.454751 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:04:43.454751 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:04:43.454751 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:04:43.454751 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:04:43.454751 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:04:43.454751 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:04:43.454751 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 01:04:43.455644 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 01:04:43.455644 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 01:04:43.455644 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Aug 31 01:04:43.457856 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 364/752] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 01:04:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:04:43.458868 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b6f424-e816-407b-a9ab-1799ced920d4 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:43.506759 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4be74f8-9ea0-46b8-b797-5418110b8316 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:43.530265 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:43.545656 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca5b7d7-a543-4f8b-a56d-2cb134275fc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:43.546328 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca5b7d7-a543-4f8b-a56d-2cb134275fc9 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:43.546611 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca5b7d7-a543-4f8b-a56d-2cb134275fc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:43.546885 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca5b7d7-a543-4f8b-a56d-2cb134275fc9 None admin] PATH_INFO: `/v3/projects` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:43.547600 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca5b7d7-a543-4f8b-a56d-2cb134275fc9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:04:43.548062 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca5b7d7-a543-4f8b-a56d-2cb134275fc9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:43.549426 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca5b7d7-a543-4f8b-a56d-2cb134275fc9 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:43.563267 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 389/753] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Aug 31 01:04:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:04:43.564153 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca5b7d7-a543-4f8b-a56d-2cb134275fc9 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:43.608662 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b6f424-e816-407b-a9ab-1799ced920d4 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:43.645489 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470f3cdc-f003-442a-b5d9-3428d31bd7fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:43.646135 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470f3cdc-f003-442a-b5d9-3428d31bd7fc None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:43.646418 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470f3cdc-f003-442a-b5d9-3428d31bd7fc None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:43.646707 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470f3cdc-f003-442a-b5d9-3428d31bd7fc None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:43.654593 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470f3cdc-f003-442a-b5d9-3428d31bd7fc None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:43.656052 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470f3cdc-f003-442a-b5d9-3428d31bd7fc None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:43.660490 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:43.668153 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application [None req-470f3cdc-f003-442a-b5d9-3428d31bd7fc None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 31 01:04:43.668153 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:04:43.668153 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:04:43.668153 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:04:43.668153 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:04:43.668153 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:04:43.668153 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:04:43.668153 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:04:43.668153 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:04:43.668153 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:04:43.668153 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:04:43.668153 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:04:43.668153 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 01:04:43.668153 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 01:04:43.668153 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 01:04:43.668153 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 01:04:43.681173 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:04:43.681173 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:04:43.681173 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 01:04:43.681173 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 01:04:43.681173 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 01:04:43.681173 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 01:04:43.681173 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:04:43.681173 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:04:43.681173 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:04:43.681173 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:04:43.681173 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:04:43.681173 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application with Lock( Aug 31 01:04:43.681173 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:04:43.681173 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:04:43.681173 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:04:43.681173 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:04:43.681173 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:04:43.681173 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:04:43.682269 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:04:43.682269 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:04:43.682269 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 01:04:43.682269 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 01:04:43.682269 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 01:04:43.682269 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 01:04:43.682269 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 01:04:43.682269 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 01:04:43.682269 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 31 01:04:43.682269 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Aug 31 01:04:43.682269 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 365/754] 10.222.0.20 () {56 vars in 1117 bytes} [Wed Aug 31 01:04:43 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:04:43.682269 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470f3cdc-f003-442a-b5d9-3428d31bd7fc None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:43.693220 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca5b7d7-a543-4f8b-a56d-2cb134275fc9 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:43.732068 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8f08b2-4fd7-4101-97e1-fcbe4bf8a752 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:43.733172 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8f08b2-4fd7-4101-97e1-fcbe4bf8a752 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:43.733776 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8f08b2-4fd7-4101-97e1-fcbe4bf8a752 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:43.734252 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8f08b2-4fd7-4101-97e1-fcbe4bf8a752 None admin] PATH_INFO: `/v3/domains` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:43.735087 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8f08b2-4fd7-4101-97e1-fcbe4bf8a752 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:04:43.735603 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8f08b2-4fd7-4101-97e1-fcbe4bf8a752 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:43.737383 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8f08b2-4fd7-4101-97e1-fcbe4bf8a752 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:43.751786 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 390/755] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 31 01:04:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:04:43.753014 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8f08b2-4fd7-4101-97e1-fcbe4bf8a752 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:43.808810 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470f3cdc-f003-442a-b5d9-3428d31bd7fc None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:43.811207 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:43.858351 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ed006e-958e-46ad-b443-3417ee38697a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:43.858960 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ed006e-958e-46ad-b443-3417ee38697a None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:43.859237 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ed006e-958e-46ad-b443-3417ee38697a None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:43.859672 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ed006e-958e-46ad-b443-3417ee38697a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:43.860535 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ed006e-958e-46ad-b443-3417ee38697a None admin] RBAC: Adding query filter params (role.id=d009411496b8452b88e77ffb8db015a7, scope.project.id=60b2abc15c2849ba8a1ed92117cc981b, user.id=fc1d010919ec40878ae829cc8dceec4c) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:04:43.860896 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ed006e-958e-46ad-b443-3417ee38697a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:43.863079 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ed006e-958e-46ad-b443-3417ee38697a None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:43.868595 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8f08b2-4fd7-4101-97e1-fcbe4bf8a752 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:43.870612 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 366/756] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Aug 31 01:04:43 2022] GET /identity/v3/role_assignments?user.id=fc1d010919ec40878ae829cc8dceec4c&scope.project.id=60b2abc15c2849ba8a1ed92117cc981b&role.id=d009411496b8452b88e77ffb8db015a7 => generated 259 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:04:43.871431 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ed006e-958e-46ad-b443-3417ee38697a None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:43.938115 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef523c4-be59-413b-969e-393ee446ae02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:43.939104 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef523c4-be59-413b-969e-393ee446ae02 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:43.939610 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef523c4-be59-413b-969e-393ee446ae02 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:43.940106 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef523c4-be59-413b-969e-393ee446ae02 None admin] PATH_INFO: `/v3/users` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:43.941072 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef523c4-be59-413b-969e-393ee446ae02 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:04:43.941582 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef523c4-be59-413b-969e-393ee446ae02 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:43.946548 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef523c4-be59-413b-969e-393ee446ae02 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:43.954213 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:43.982522 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 391/757] 10.222.0.20 () {56 vars in 1149 bytes} [Wed Aug 31 01:04:43 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:04:43.989584 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef523c4-be59-413b-969e-393ee446ae02 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:44.003461 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ed006e-958e-46ad-b443-3417ee38697a None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:44.045775 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665a545d-dc6c-4a63-8811-4a69b9b48278 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:44.046660 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665a545d-dc6c-4a63-8811-4a69b9b48278 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:44.046947 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665a545d-dc6c-4a63-8811-4a69b9b48278 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:44.047218 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665a545d-dc6c-4a63-8811-4a69b9b48278 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:44.056144 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665a545d-dc6c-4a63-8811-4a69b9b48278 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:44.057763 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665a545d-dc6c-4a63-8811-4a69b9b48278 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:44.073622 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application [None req-665a545d-dc6c-4a63-8811-4a69b9b48278 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 01:04:44.073622 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:04:44.073622 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:04:44.073622 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:04:44.073622 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:04:44.073622 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:04:44.073622 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:04:44.073622 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:04:44.073622 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:04:44.073622 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:04:44.073622 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:04:44.073622 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:04:44.073622 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 01:04:44.073622 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 01:04:44.073622 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 01:04:44.074808 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 01:04:44.074808 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:04:44.074808 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:04:44.074808 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:04:44.074808 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:04:44.074808 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:04:44.074808 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:04:44.074808 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:04:44.074808 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application with Lock( Aug 31 01:04:44.074808 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:04:44.074808 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:04:44.074808 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:04:44.074808 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:04:44.074808 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:04:44.074808 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:04:44.074808 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:04:44.074808 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:04:44.076099 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 01:04:44.076099 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 01:04:44.076099 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 01:04:44.076099 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 01:04:44.076099 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 01:04:44.076099 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 01:04:44.076099 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 01:04:44.076099 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Aug 31 01:04:44.076099 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 367/758] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Aug 31 01:04:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:04:44.081364 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665a545d-dc6c-4a63-8811-4a69b9b48278 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:44.117127 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:44.184113 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef523c4-be59-413b-969e-393ee446ae02 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:44.211651 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665a545d-dc6c-4a63-8811-4a69b9b48278 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:44.225241 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4cdd46-27c0-43e7-9db9-96cafe500e02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:44.225981 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4cdd46-27c0-43e7-9db9-96cafe500e02 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:44.226265 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4cdd46-27c0-43e7-9db9-96cafe500e02 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:44.226541 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4cdd46-27c0-43e7-9db9-96cafe500e02 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:44.262083 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01403cec-75fd-42e9-950e-cc7025020bf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:44.262686 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01403cec-75fd-42e9-950e-cc7025020bf1 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:44.262968 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01403cec-75fd-42e9-950e-cc7025020bf1 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:44.263244 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01403cec-75fd-42e9-950e-cc7025020bf1 None admin] PATH_INFO: `/v3/projects` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:44.263941 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01403cec-75fd-42e9-950e-cc7025020bf1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:04:44.264247 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01403cec-75fd-42e9-950e-cc7025020bf1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:44.266587 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01403cec-75fd-42e9-950e-cc7025020bf1 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:44.284785 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 368/759] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Aug 31 01:04:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:04:44.286753 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ccacfc-1fa1-4305-9e7e-631f0242166b None None] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:44.287395 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ccacfc-1fa1-4305-9e7e-631f0242166b None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:44.291685 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ccacfc-1fa1-4305-9e7e-631f0242166b None None] PATH_INFO: `/` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:44.293460 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 369/760] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:04:44 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:04:44.294400 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ccacfc-1fa1-4305-9e7e-631f0242166b None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:44.307704 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:44.315596 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4cdd46-27c0-43e7-9db9-96cafe500e02 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:44.317339 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4cdd46-27c0-43e7-9db9-96cafe500e02 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:44.440431 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:44.462078 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 392/761] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Aug 31 01:04:43 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 473 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 01:04:44.464636 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ea0819-c798-41a9-b3f6-0bad0f909f6b None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:44.465454 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ea0819-c798-41a9-b3f6-0bad0f909f6b None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:44.466215 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ea0819-c798-41a9-b3f6-0bad0f909f6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:44.471994 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ccacfc-1fa1-4305-9e7e-631f0242166b None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:44.496865 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-d0ea0819-c798-41a9-b3f6-0bad0f909f6b None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:04:44.499229 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-d0ea0819-c798-41a9-b3f6-0bad0f909f6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:04:44.535904 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581c7425-b668-4b58-9f79-fc02d26948dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:44.536980 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581c7425-b668-4b58-9f79-fc02d26948dd None admin] REQUEST_METHOD: `PUT` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:44.537454 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581c7425-b668-4b58-9f79-fc02d26948dd None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:44.537923 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581c7425-b668-4b58-9f79-fc02d26948dd None admin] PATH_INFO: `/v3/projects/60b2abc15c2849ba8a1ed92117cc981b/users/8744c9de803048f583d0317d01f6a540/roles/d009411496b8452b88e77ffb8db015a7` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:44.541937 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581c7425-b668-4b58-9f79-fc02d26948dd None admin] RBAC: Authorizing `identity:create_grant(project_id=60b2abc15c2849ba8a1ed92117cc981b, user_id=8744c9de803048f583d0317d01f6a540, role_id=d009411496b8452b88e77ffb8db015a7)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:44.544659 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581c7425-b668-4b58-9f79-fc02d26948dd None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:44.597103 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 393/762] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:04:44 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 01:04:44.598876 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ea0819-c798-41a9-b3f6-0bad0f909f6b None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:44.619251 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 370/763] 10.222.0.20 () {58 vars in 1439 bytes} [Wed Aug 31 01:04:44 2022] PUT /identity/v3/projects/60b2abc15c2849ba8a1ed92117cc981b/users/8744c9de803048f583d0317d01f6a540/roles/d009411496b8452b88e77ffb8db015a7 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:04:44.636010 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:44.717814 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ea0819-c798-41a9-b3f6-0bad0f909f6b None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:44.755134 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02557ca-a4e3-4419-b4e9-240e690ee246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:44.755790 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02557ca-a4e3-4419-b4e9-240e690ee246 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:44.756072 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02557ca-a4e3-4419-b4e9-240e690ee246 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:44.756350 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02557ca-a4e3-4419-b4e9-240e690ee246 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:44.768030 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02557ca-a4e3-4419-b4e9-240e690ee246 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:44.769571 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02557ca-a4e3-4419-b4e9-240e690ee246 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:44.786734 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application [None req-c02557ca-a4e3-4419-b4e9-240e690ee246 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 01:04:44.786734 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:04:44.786734 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:04:44.786734 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:04:44.786734 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:04:44.786734 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:04:44.786734 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:04:44.786734 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:04:44.786734 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:04:44.786734 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:04:44.786734 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:04:44.786734 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:04:44.786734 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 01:04:44.786734 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 01:04:44.786734 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 01:04:44.786734 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 01:04:44.787748 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:04:44.787748 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:04:44.787748 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:04:44.787748 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:04:44.787748 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:04:44.787748 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:04:44.787748 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:04:44.787748 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application with Lock( Aug 31 01:04:44.787748 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:04:44.787748 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:04:44.787748 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:04:44.787748 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:04:44.787748 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:04:44.787748 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:04:44.787748 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:04:44.787748 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:04:44.787748 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 01:04:44.788676 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 01:04:44.788676 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 01:04:44.788676 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Aug 31 01:04:44.788676 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 394/764] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 01:04:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:04:44.795001 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581c7425-b668-4b58-9f79-fc02d26948dd None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:44.933997 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581c7425-b668-4b58-9f79-fc02d26948dd None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:44.972231 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ac6f19-0076-48a3-a4bf-1dd5d75b81e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:44.972791 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ac6f19-0076-48a3-a4bf-1dd5d75b81e8 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:44.973079 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ac6f19-0076-48a3-a4bf-1dd5d75b81e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:44.973358 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ac6f19-0076-48a3-a4bf-1dd5d75b81e8 None admin] PATH_INFO: `/v3/domains` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:44.974005 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ac6f19-0076-48a3-a4bf-1dd5d75b81e8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:04:44.974314 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ac6f19-0076-48a3-a4bf-1dd5d75b81e8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:44.975724 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ac6f19-0076-48a3-a4bf-1dd5d75b81e8 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:44.980355 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02557ca-a4e3-4419-b4e9-240e690ee246 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:44.988143 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 371/765] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 31 01:04:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:04:44.994037 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ac6f19-0076-48a3-a4bf-1dd5d75b81e8 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:45.006825 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:45.093307 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:45.139422 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02557ca-a4e3-4419-b4e9-240e690ee246 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:45.178440 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12da8bf-7173-4505-a644-42431c24008e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:45.179115 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12da8bf-7173-4505-a644-42431c24008e None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:45.179424 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12da8bf-7173-4505-a644-42431c24008e None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:45.179666 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12da8bf-7173-4505-a644-42431c24008e None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:45.180251 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12da8bf-7173-4505-a644-42431c24008e None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:45.181663 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12da8bf-7173-4505-a644-42431c24008e None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:45.186872 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ac6f19-0076-48a3-a4bf-1dd5d75b81e8 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:45.190703 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application [None req-d12da8bf-7173-4505-a644-42431c24008e None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 31 01:04:45.190703 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:04:45.190703 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 01:04:45.190703 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 01:04:45.190703 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 01:04:45.190703 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 01:04:45.190703 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 01:04:45.190703 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 01:04:45.190703 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 31 01:04:45.190703 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Aug 31 01:04:45.190703 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 01:04:45.190703 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Aug 31 01:04:45.190703 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:04:45.190703 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:04:45.190703 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:04:45.190703 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:04:45.190703 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:04:45.191662 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:04:45.191662 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:04:45.191662 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:04:45.191662 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:04:45.191662 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:04:45.191662 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:04:45.191662 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 01:04:45.191662 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 01:04:45.191662 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 01:04:45.191662 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 01:04:45.191662 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:04:45.191662 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:04:45.191662 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:04:45.191662 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:04:45.191662 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:04:45.191662 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:04:45.192477 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:04:45.192477 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application with Lock( Aug 31 01:04:45.192477 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:04:45.192477 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:04:45.192477 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:04:45.192477 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:04:45.192477 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:04:45.192477 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:04:45.192477 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:04:45.192477 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:04:45.192477 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 01:04:45.192477 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 01:04:45.192477 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 31 01:04:45.192477 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Aug 31 01:04:45.192477 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 395/766] 10.222.0.20 () {56 vars in 1129 bytes} [Wed Aug 31 01:04:44 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 01:04:45.201732 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12da8bf-7173-4505-a644-42431c24008e None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:45.256597 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e628169a-9785-48bb-85b5-d18d10b465dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:45.257297 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e628169a-9785-48bb-85b5-d18d10b465dd None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:45.257586 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e628169a-9785-48bb-85b5-d18d10b465dd None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:45.257869 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e628169a-9785-48bb-85b5-d18d10b465dd None admin] PATH_INFO: `/v3/projects/service` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:45.265454 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e628169a-9785-48bb-85b5-d18d10b465dd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:45.266939 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e628169a-9785-48bb-85b5-d18d10b465dd None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:45.279492 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application [None req-e628169a-9785-48bb-85b5-d18d10b465dd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 01:04:45.279492 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:04:45.279492 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:04:45.279492 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:04:45.279492 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:04:45.279492 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:04:45.279492 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:04:45.279492 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:04:45.279492 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:04:45.279492 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:04:45.279492 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:04:45.279492 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:04:45.279492 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 01:04:45.279492 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 01:04:45.279492 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 01:04:45.280432 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 01:04:45.280432 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:04:45.280432 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:04:45.280432 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:04:45.280432 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:04:45.280432 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:04:45.280432 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:04:45.280432 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:04:45.280432 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application with Lock( Aug 31 01:04:45.280432 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:04:45.280432 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:04:45.280432 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:04:45.280432 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:04:45.280432 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:04:45.280432 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:04:45.280432 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:04:45.280432 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:04:45.281344 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 01:04:45.281344 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 01:04:45.281344 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 01:04:45.281344 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 01:04:45.281344 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 01:04:45.281344 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 01:04:45.281344 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 01:04:45.281344 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Aug 31 01:04:45.285329 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 372/767] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Aug 31 01:04:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 01:04:45.286609 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4935ae1a-7a48-4a4b-b7e7-c4620a086956 None None] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:45.286896 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4935ae1a-7a48-4a4b-b7e7-c4620a086956 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:45.287170 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4935ae1a-7a48-4a4b-b7e7-c4620a086956 None None] PATH_INFO: `/` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:45.289719 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 373/768] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:04:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:04:45.290566 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4935ae1a-7a48-4a4b-b7e7-c4620a086956 None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:45.305409 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12da8bf-7173-4505-a644-42431c24008e None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:45.339935 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5883d286-f2c9-4d9d-9277-124536cfd364 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:45.340538 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5883d286-f2c9-4d9d-9277-124536cfd364 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:45.340845 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5883d286-f2c9-4d9d-9277-124536cfd364 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:45.341121 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5883d286-f2c9-4d9d-9277-124536cfd364 None admin] PATH_INFO: `/v3/services` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:45.341744 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5883d286-f2c9-4d9d-9277-124536cfd364 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:04:45.342070 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5883d286-f2c9-4d9d-9277-124536cfd364 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:45.343413 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5883d286-f2c9-4d9d-9277-124536cfd364 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:45.348286 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 396/769] 10.222.0.20 () {56 vars in 1124 bytes} [Wed Aug 31 01:04:45 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:04:45.349612 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d635733a-167e-4879-808b-999e4fca861b None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:45.349896 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d635733a-167e-4879-808b-999e4fca861b None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:45.350173 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d635733a-167e-4879-808b-999e4fca861b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:45.360011 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:45.369965 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-d635733a-167e-4879-808b-999e4fca861b None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:04:45.371503 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-d635733a-167e-4879-808b-999e4fca861b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:04:45.381530 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:45.395830 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4935ae1a-7a48-4a4b-b7e7-c4620a086956 None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:45.432424 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc8ffe1-21f0-428c-acd8-2377a7394cb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:45.433428 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc8ffe1-21f0-428c-acd8-2377a7394cb5 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:45.433904 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc8ffe1-21f0-428c-acd8-2377a7394cb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:45.434322 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc8ffe1-21f0-428c-acd8-2377a7394cb5 None admin] PATH_INFO: `/v3/projects` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:45.435347 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc8ffe1-21f0-428c-acd8-2377a7394cb5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:04:45.435956 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc8ffe1-21f0-428c-acd8-2377a7394cb5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:45.440236 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc8ffe1-21f0-428c-acd8-2377a7394cb5 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:45.449438 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 397/770] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:04:45 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 01:04:45.450041 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d635733a-167e-4879-808b-999e4fca861b None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:45.467676 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 374/771] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Aug 31 01:04:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:04:45.473246 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc8ffe1-21f0-428c-acd8-2377a7394cb5 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:45.559836 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d635733a-167e-4879-808b-999e4fca861b None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:45.605841 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f6fca5-72f4-49d3-ab25-0590ce7a65cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:45.606462 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f6fca5-72f4-49d3-ab25-0590ce7a65cd None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:45.606746 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f6fca5-72f4-49d3-ab25-0590ce7a65cd None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:45.607022 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f6fca5-72f4-49d3-ab25-0590ce7a65cd None admin] PATH_INFO: `/v3/services` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:45.607652 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f6fca5-72f4-49d3-ab25-0590ce7a65cd None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:04:45.607958 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f6fca5-72f4-49d3-ab25-0590ce7a65cd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:45.609353 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f6fca5-72f4-49d3-ab25-0590ce7a65cd None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:45.616885 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:45.623636 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 398/772] 10.222.0.20 () {56 vars in 1124 bytes} [Wed Aug 31 01:04:45 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:04:45.629856 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f6fca5-72f4-49d3-ab25-0590ce7a65cd None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:45.701082 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc8ffe1-21f0-428c-acd8-2377a7394cb5 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:45.737295 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:45.754693 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8954d6-f1c2-4132-a32a-00fa1d94d1d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:45.755303 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8954d6-f1c2-4132-a32a-00fa1d94d1d6 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:45.755579 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8954d6-f1c2-4132-a32a-00fa1d94d1d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:45.755851 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8954d6-f1c2-4132-a32a-00fa1d94d1d6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:45.756635 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8954d6-f1c2-4132-a32a-00fa1d94d1d6 None admin] RBAC: Adding query filter params (role.id=7d00a4cfa13744a69604ac2cf1a5816e, scope.project.id=60b2abc15c2849ba8a1ed92117cc981b, user.id=b3efb4f9daac4a0ca05a36dd361f17e5) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:04:45.758372 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8954d6-f1c2-4132-a32a-00fa1d94d1d6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:45.759828 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8954d6-f1c2-4132-a32a-00fa1d94d1d6 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:45.767561 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 375/773] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Aug 31 01:04:45 2022] GET /identity/v3/role_assignments?user.id=b3efb4f9daac4a0ca05a36dd361f17e5&scope.project.id=60b2abc15c2849ba8a1ed92117cc981b&role.id=7d00a4cfa13744a69604ac2cf1a5816e => generated 259 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:04:45.768646 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8954d6-f1c2-4132-a32a-00fa1d94d1d6 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:45.811813 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f6fca5-72f4-49d3-ab25-0590ce7a65cd None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:45.907785 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8954d6-f1c2-4132-a32a-00fa1d94d1d6 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:45.913548 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703af2b2-bff9-48fa-bd1e-e446b2133621 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:45.914185 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703af2b2-bff9-48fa-bd1e-e446b2133621 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:45.914467 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703af2b2-bff9-48fa-bd1e-e446b2133621 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:45.914741 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703af2b2-bff9-48fa-bd1e-e446b2133621 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:45.915455 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703af2b2-bff9-48fa-bd1e-e446b2133621 None admin] RBAC: Adding query filter params (interface=public, service_id=d036215779c54a4db89a870af04aebbd, region_id=RegionOne) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:04:45.915832 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703af2b2-bff9-48fa-bd1e-e446b2133621 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:45.917268 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703af2b2-bff9-48fa-bd1e-e446b2133621 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:45.942706 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 399/774] 10.222.0.20 () {56 vars in 1251 bytes} [Wed Aug 31 01:04:45 2022] GET /identity/v3/endpoints?service_id=d036215779c54a4db89a870af04aebbd&interface=public®ion_id=RegionOne => generated 194 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:04:45.945406 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2375ca22-7b94-449e-ad0a-278d109f7076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:45.946058 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2375ca22-7b94-449e-ad0a-278d109f7076 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:45.946340 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2375ca22-7b94-449e-ad0a-278d109f7076 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:45.946617 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2375ca22-7b94-449e-ad0a-278d109f7076 None admin] PATH_INFO: `/v3/services/network` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:45.947212 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2375ca22-7b94-449e-ad0a-278d109f7076 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:45.948590 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2375ca22-7b94-449e-ad0a-278d109f7076 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:45.950631 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703af2b2-bff9-48fa-bd1e-e446b2133621 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:45.962531 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application [None req-2375ca22-7b94-449e-ad0a-278d109f7076 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 31 01:04:45.962531 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:04:45.962531 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 01:04:45.962531 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 01:04:45.962531 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 01:04:45.962531 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 01:04:45.962531 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 01:04:45.962531 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 01:04:45.962531 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 31 01:04:45.962531 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Aug 31 01:04:45.962531 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 01:04:45.962531 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Aug 31 01:04:45.962531 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:04:45.962531 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:04:45.962531 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:04:45.962531 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:04:45.962531 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:04:45.963580 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:04:45.963580 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:04:45.963580 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:04:45.963580 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:04:45.963580 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:04:45.963580 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:04:45.963580 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 01:04:45.963580 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 01:04:45.963580 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 01:04:45.963580 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 01:04:45.963580 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:04:45.963580 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:04:45.963580 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:04:45.963580 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:04:45.963580 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:04:45.963580 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:04:45.964455 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:04:45.964455 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application with Lock( Aug 31 01:04:45.964455 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:04:45.964455 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:04:45.964455 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:04:45.964455 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:04:45.964455 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:04:45.964455 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:04:45.964455 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:04:45.964455 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:04:45.964455 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 01:04:45.964455 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 01:04:45.964455 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 31 01:04:45.964455 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Aug 31 01:04:45.970878 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 376/775] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Aug 31 01:04:45 2022] GET /identity/v3/services/network => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:04:45.972634 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2375ca22-7b94-449e-ad0a-278d109f7076 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:46.045613 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:46.059651 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:46.096324 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703af2b2-bff9-48fa-bd1e-e446b2133621 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:46.165732 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5546ec3d-44f2-436c-864a-b281c7075a9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:46.168225 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5546ec3d-44f2-436c-864a-b281c7075a9f None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:46.168716 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5546ec3d-44f2-436c-864a-b281c7075a9f None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:46.170105 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5546ec3d-44f2-436c-864a-b281c7075a9f None admin] PATH_INFO: `/v3/services` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:46.170918 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5546ec3d-44f2-436c-864a-b281c7075a9f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:46.173768 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5546ec3d-44f2-436c-864a-b281c7075a9f None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:46.181547 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2375ca22-7b94-449e-ad0a-278d109f7076 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:46.187068 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 400/776] 10.222.0.20 () {56 vars in 1087 bytes} [Wed Aug 31 01:04:45 2022] GET /identity/v3/services => generated 772 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 01:04:46.196085 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06031f3b-299d-481a-be9f-89a48274fc62 None None] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:46.196410 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06031f3b-299d-481a-be9f-89a48274fc62 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:46.196648 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06031f3b-299d-481a-be9f-89a48274fc62 None None] PATH_INFO: `/` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:46.198362 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 401/777] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:04:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 01:04:46.204653 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d42fa8-0eec-4aec-9ebd-bc808d40379e None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:46.205079 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d42fa8-0eec-4aec-9ebd-bc808d40379e None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:46.205654 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d42fa8-0eec-4aec-9ebd-bc808d40379e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:46.235260 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-87d42fa8-0eec-4aec-9ebd-bc808d40379e None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:04:46.239617 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-87d42fa8-0eec-4aec-9ebd-bc808d40379e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:04:46.244019 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e821e0-d3e3-4f54-9c4f-d92f65b96a19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:46.244981 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e821e0-d3e3-4f54-9c4f-d92f65b96a19 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:46.245476 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e821e0-d3e3-4f54-9c4f-d92f65b96a19 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:46.245953 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e821e0-d3e3-4f54-9c4f-d92f65b96a19 None admin] PATH_INFO: `/v3/services` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:46.246905 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e821e0-d3e3-4f54-9c4f-d92f65b96a19 None admin] RBAC: Adding query filter params (name=network) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:04:46.247428 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e821e0-d3e3-4f54-9c4f-d92f65b96a19 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:46.249251 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e821e0-d3e3-4f54-9c4f-d92f65b96a19 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:46.262311 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 377/778] 10.222.0.20 () {56 vars in 1112 bytes} [Wed Aug 31 01:04:45 2022] GET /identity/v3/services?name=network => generated 124 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:04:46.268546 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e821e0-d3e3-4f54-9c4f-d92f65b96a19 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:46.388390 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 402/779] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:04:46 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 01:04:46.432959 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e821e0-d3e3-4f54-9c4f-d92f65b96a19 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:46.454598 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:46.472223 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b027960a-54ef-452c-acef-5ee4306f8f4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:46.472878 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b027960a-54ef-452c-acef-5ee4306f8f4b None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:46.473173 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b027960a-54ef-452c-acef-5ee4306f8f4b None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:46.473447 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b027960a-54ef-452c-acef-5ee4306f8f4b None admin] PATH_INFO: `/v3/services` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:46.474093 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b027960a-54ef-452c-acef-5ee4306f8f4b None admin] RBAC: Adding query filter params (type=network) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:04:46.474401 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b027960a-54ef-452c-acef-5ee4306f8f4b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:46.476334 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b027960a-54ef-452c-acef-5ee4306f8f4b None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:46.481772 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 378/780] 10.222.0.20 () {56 vars in 1112 bytes} [Wed Aug 31 01:04:46 2022] GET /identity/v3/services?type=network => generated 350 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:04:46.487451 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d42fa8-0eec-4aec-9ebd-bc808d40379e None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:46.580226 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:46.639726 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d42fa8-0eec-4aec-9ebd-bc808d40379e None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:46.685971 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fecb828-06e1-4208-9624-e7fd0137a19d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:46.686592 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fecb828-06e1-4208-9624-e7fd0137a19d None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:46.686876 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fecb828-06e1-4208-9624-e7fd0137a19d None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:46.687156 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fecb828-06e1-4208-9624-e7fd0137a19d None admin] PATH_INFO: `/v3/endpoints` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:46.687857 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fecb828-06e1-4208-9624-e7fd0137a19d None admin] RBAC: Adding query filter params (interface=public, service_id=7f101e1a20ac4af1ba780cfe3c587b4f, region_id=RegionOne) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:04:46.688168 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fecb828-06e1-4208-9624-e7fd0137a19d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:46.689559 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fecb828-06e1-4208-9624-e7fd0137a19d None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:46.698354 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 403/781] 10.222.0.20 () {56 vars in 1251 bytes} [Wed Aug 31 01:04:46 2022] GET /identity/v3/endpoints?service_id=7f101e1a20ac4af1ba780cfe3c587b4f&interface=public®ion_id=RegionOne => generated 194 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:04:46.704368 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b027960a-54ef-452c-acef-5ee4306f8f4b None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:46.751546 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fecb828-06e1-4208-9624-e7fd0137a19d None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:46.835917 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b027960a-54ef-452c-acef-5ee4306f8f4b None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:46.861061 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fecb828-06e1-4208-9624-e7fd0137a19d None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:46.886467 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:46.907073 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc63b0f-5b35-4a07-9cdb-04866ab302f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:46.907720 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc63b0f-5b35-4a07-9cdb-04866ab302f3 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:46.912221 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc63b0f-5b35-4a07-9cdb-04866ab302f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:46.912221 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc63b0f-5b35-4a07-9cdb-04866ab302f3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:46.913472 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae56f4c5-4355-4a61-a6ef-b4704b527294 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:46.914094 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae56f4c5-4355-4a61-a6ef-b4704b527294 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:46.914377 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae56f4c5-4355-4a61-a6ef-b4704b527294 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:46.914654 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae56f4c5-4355-4a61-a6ef-b4704b527294 None admin] PATH_INFO: `/v3/services` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:46.915268 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae56f4c5-4355-4a61-a6ef-b4704b527294 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:46.916650 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae56f4c5-4355-4a61-a6ef-b4704b527294 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:46.919871 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc63b0f-5b35-4a07-9cdb-04866ab302f3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 01:04:46.924810 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application [None req-cfc63b0f-5b35-4a07-9cdb-04866ab302f3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 01:04:46.924810 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:04:46.924810 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:04:46.924810 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:04:46.924810 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:04:46.924810 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:04:46.924810 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:04:46.924810 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:04:46.924810 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:04:46.924810 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:04:46.924810 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:04:46.924810 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:04:46.924810 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 01:04:46.924810 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 01:04:46.924810 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 01:04:46.924810 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 01:04:46.925746 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 01:04:46.925746 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 01:04:46.925746 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 01:04:46.925746 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 01:04:46.925746 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:04:46.925746 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:04:46.925746 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:04:46.925746 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:04:46.925746 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:04:46.925746 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:04:46.925746 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:04:46.925746 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application with Lock( Aug 31 01:04:46.925746 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:04:46.925746 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:04:46.925746 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:04:46.925746 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:04:46.926592 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:04:46.926592 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:04:46.926592 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:04:46.926592 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:04:46.926592 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 01:04:46.926592 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 01:04:46.926592 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 01:04:46.926592 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 01:04:46.926592 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 01:04:46.926592 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 01:04:46.926592 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 01:04:46.926592 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Aug 31 01:04:46.926592 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 404/782] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Aug 31 01:04:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:04:46.928878 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 379/783] 10.222.0.20 () {56 vars in 1087 bytes} [Wed Aug 31 01:04:46 2022] GET /identity/v3/services => generated 772 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:04:46.931769 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc63b0f-5b35-4a07-9cdb-04866ab302f3 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:47.067328 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad3269e-fc4c-4941-8c6b-8afa35538ed4 None None] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:47.067680 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad3269e-fc4c-4941-8c6b-8afa35538ed4 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:47.067894 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad3269e-fc4c-4941-8c6b-8afa35538ed4 None None] PATH_INFO: `/` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:47.070062 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 380/784] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:04:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:04:47.077658 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3296ba-bec5-4c0a-8b69-e29f6cc55894 None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:47.077960 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3296ba-bec5-4c0a-8b69-e29f6cc55894 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:47.078216 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3296ba-bec5-4c0a-8b69-e29f6cc55894 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:47.098048 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc63b0f-5b35-4a07-9cdb-04866ab302f3 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:47.113004 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-9d3296ba-bec5-4c0a-8b69-e29f6cc55894 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:04:47.114258 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-9d3296ba-bec5-4c0a-8b69-e29f6cc55894 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:04:47.176239 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce134ea-696e-42bb-ab0c-c4facb97d146 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:47.176923 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce134ea-696e-42bb-ab0c-c4facb97d146 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:47.177209 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce134ea-696e-42bb-ab0c-c4facb97d146 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:47.177487 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce134ea-696e-42bb-ab0c-c4facb97d146 None admin] PATH_INFO: `/v3/roles` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:47.178233 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce134ea-696e-42bb-ab0c-c4facb97d146 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:04:47.178545 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce134ea-696e-42bb-ab0c-c4facb97d146 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:47.180007 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce134ea-696e-42bb-ab0c-c4facb97d146 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:47.183333 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:47.200679 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 405/785] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Aug 31 01:04:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:04:47.207028 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce134ea-696e-42bb-ab0c-c4facb97d146 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:47.215823 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 381/786] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:04:47 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 01:04:47.266928 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:47.326454 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce134ea-696e-42bb-ab0c-c4facb97d146 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:47.367788 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6114fe5-ae8c-4eeb-a897-a795fd0eb37d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:47.368430 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6114fe5-ae8c-4eeb-a897-a795fd0eb37d None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:47.368714 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6114fe5-ae8c-4eeb-a897-a795fd0eb37d None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:47.369371 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6114fe5-ae8c-4eeb-a897-a795fd0eb37d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:47.385214 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6114fe5-ae8c-4eeb-a897-a795fd0eb37d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:47.386911 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6114fe5-ae8c-4eeb-a897-a795fd0eb37d None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:47.401833 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application [None req-e6114fe5-ae8c-4eeb-a897-a795fd0eb37d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 01:04:47.401833 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:04:47.401833 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:04:47.401833 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:04:47.401833 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:04:47.401833 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:04:47.401833 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:04:47.401833 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:04:47.401833 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:04:47.401833 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:04:47.401833 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:04:47.401833 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:04:47.401833 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 01:04:47.401833 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 01:04:47.401833 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 01:04:47.401833 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 01:04:47.402973 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:04:47.402973 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:04:47.402973 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:04:47.402973 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:04:47.402973 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:04:47.402973 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:04:47.402973 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:04:47.402973 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application with Lock( Aug 31 01:04:47.402973 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:04:47.402973 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:04:47.402973 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:04:47.402973 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:04:47.402973 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:04:47.402973 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:04:47.402973 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:04:47.402973 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:04:47.402973 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 01:04:47.404088 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 01:04:47.404088 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 01:04:47.404088 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Aug 31 01:04:47.409956 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 406/787] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 01:04:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 01:04:47.416291 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3296ba-bec5-4c0a-8b69-e29f6cc55894 None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:47.546971 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6114fe5-ae8c-4eeb-a897-a795fd0eb37d None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:47.565999 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3296ba-bec5-4c0a-8b69-e29f6cc55894 None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:47.612171 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a9e058-5665-46e2-93c3-0fb634b256ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:47.613094 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a9e058-5665-46e2-93c3-0fb634b256ba None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:47.613558 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a9e058-5665-46e2-93c3-0fb634b256ba None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:47.613999 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a9e058-5665-46e2-93c3-0fb634b256ba None admin] PATH_INFO: `/v3/domains` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:47.614827 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a9e058-5665-46e2-93c3-0fb634b256ba None admin] RBAC: Adding query filter params (name=Default) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:04:47.615288 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a9e058-5665-46e2-93c3-0fb634b256ba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:47.617457 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a9e058-5665-46e2-93c3-0fb634b256ba None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:47.640179 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 382/788] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 31 01:04:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:04:47.645456 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a9e058-5665-46e2-93c3-0fb634b256ba None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:47.728445 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:47.815221 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:47.823225 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6114fe5-ae8c-4eeb-a897-a795fd0eb37d None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:47.844760 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a9e058-5665-46e2-93c3-0fb634b256ba None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:47.860434 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5baf078d-a3ce-45b9-826f-e347713a737b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:47.861386 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baf078d-a3ce-45b9-826f-e347713a737b None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:47.862309 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baf078d-a3ce-45b9-826f-e347713a737b None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:47.862734 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baf078d-a3ce-45b9-826f-e347713a737b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:47.873720 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5baf078d-a3ce-45b9-826f-e347713a737b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 01:04:47.878828 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application [None req-5baf078d-a3ce-45b9-826f-e347713a737b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 01:04:47.878828 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:04:47.878828 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:04:47.878828 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:04:47.878828 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:04:47.878828 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:04:47.878828 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:04:47.878828 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:04:47.878828 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:04:47.878828 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:04:47.878828 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:04:47.878828 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:04:47.878828 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 01:04:47.878828 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 01:04:47.878828 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 01:04:47.878828 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 01:04:47.879964 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 01:04:47.879964 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 01:04:47.879964 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 01:04:47.879964 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 01:04:47.879964 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:04:47.879964 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:04:47.879964 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:04:47.879964 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:04:47.879964 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:04:47.879964 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:04:47.879964 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:04:47.879964 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application with Lock( Aug 31 01:04:47.879964 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:04:47.879964 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:04:47.879964 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:04:47.879964 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:04:47.881040 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:04:47.881040 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:04:47.881040 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:04:47.881040 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:04:47.881040 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 01:04:47.881040 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 01:04:47.881040 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 01:04:47.881040 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 01:04:47.881040 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 01:04:47.881040 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 01:04:47.881040 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 01:04:47.881040 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Aug 31 01:04:47.883731 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 407/789] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Aug 31 01:04:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 337 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 01:04:47.889625 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5baf078d-a3ce-45b9-826f-e347713a737b None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:47.920496 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305cb175-59f6-4549-b0d0-d4ed4ebcc73c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:47.921247 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305cb175-59f6-4549-b0d0-d4ed4ebcc73c None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:47.921508 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305cb175-59f6-4549-b0d0-d4ed4ebcc73c None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:47.921782 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305cb175-59f6-4549-b0d0-d4ed4ebcc73c None admin] PATH_INFO: `/v3/users/swift` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:47.940324 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305cb175-59f6-4549-b0d0-d4ed4ebcc73c None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:47.941818 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305cb175-59f6-4549-b0d0-d4ed4ebcc73c None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:47.963903 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application [None req-305cb175-59f6-4549-b0d0-d4ed4ebcc73c None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 31 01:04:47.963903 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:04:47.963903 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:04:47.963903 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:04:47.963903 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:04:47.963903 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:04:47.963903 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:04:47.963903 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:04:47.963903 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:04:47.963903 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:04:47.963903 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:04:47.963903 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:04:47.963903 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 01:04:47.963903 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 01:04:47.963903 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 01:04:47.963903 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 01:04:47.965006 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:04:47.965006 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:04:47.965006 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 01:04:47.965006 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 01:04:47.965006 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 01:04:47.965006 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 01:04:47.965006 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:04:47.965006 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:04:47.965006 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:04:47.965006 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:04:47.965006 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:04:47.965006 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application with Lock( Aug 31 01:04:47.965006 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:04:47.965006 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:04:47.965006 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:04:47.965006 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:04:47.965006 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:04:47.965006 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:04:47.966587 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:04:47.966587 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:04:47.966587 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 01:04:47.966587 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 01:04:47.966587 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 01:04:47.966587 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 01:04:47.966587 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 01:04:47.966587 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 01:04:47.966587 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 31 01:04:47.966587 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Aug 31 01:04:47.966587 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 383/790] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Aug 31 01:04:47 2022] GET /identity/v3/users/swift => generated 83 bytes in 320 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 01:04:47.970894 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305cb175-59f6-4549-b0d0-d4ed4ebcc73c None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:48.058601 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5baf078d-a3ce-45b9-826f-e347713a737b None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:48.104655 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45db5244-032d-427c-9690-4f32f3c7f595 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:48.105221 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45db5244-032d-427c-9690-4f32f3c7f595 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:48.105504 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45db5244-032d-427c-9690-4f32f3c7f595 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:48.105777 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45db5244-032d-427c-9690-4f32f3c7f595 None admin] PATH_INFO: `/v3/roles` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:48.106440 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45db5244-032d-427c-9690-4f32f3c7f595 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:04:48.106744 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45db5244-032d-427c-9690-4f32f3c7f595 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:48.108128 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45db5244-032d-427c-9690-4f32f3c7f595 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:48.129077 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 408/791] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Aug 31 01:04:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:04:48.130702 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d2213c-185c-4bcc-84b8-ea0460ead623 None None] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:48.130989 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d2213c-185c-4bcc-84b8-ea0460ead623 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:48.131263 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d2213c-185c-4bcc-84b8-ea0460ead623 None None] PATH_INFO: `/` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:48.132691 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 409/792] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:04:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:04:48.134106 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d2213c-185c-4bcc-84b8-ea0460ead623 None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:48.147094 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:48.152141 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305cb175-59f6-4549-b0d0-d4ed4ebcc73c None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:48.155344 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:48.196206 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478e1952-0d56-4d5d-9cc7-bfdf86bb3458 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:48.196994 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478e1952-0d56-4d5d-9cc7-bfdf86bb3458 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:48.197281 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478e1952-0d56-4d5d-9cc7-bfdf86bb3458 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:48.197847 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478e1952-0d56-4d5d-9cc7-bfdf86bb3458 None admin] PATH_INFO: `/v3/users` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:48.198598 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478e1952-0d56-4d5d-9cc7-bfdf86bb3458 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:04:48.198907 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478e1952-0d56-4d5d-9cc7-bfdf86bb3458 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:48.201129 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478e1952-0d56-4d5d-9cc7-bfdf86bb3458 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:48.231367 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 384/793] 10.222.0.20 () {56 vars in 1135 bytes} [Wed Aug 31 01:04:47 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:04:48.232855 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79c94af-c122-4094-9306-256d367895c5 None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:48.233148 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79c94af-c122-4094-9306-256d367895c5 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:48.233885 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79c94af-c122-4094-9306-256d367895c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:48.257131 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-c79c94af-c122-4094-9306-256d367895c5 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:04:48.258978 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-c79c94af-c122-4094-9306-256d367895c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:04:48.261490 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d2213c-185c-4bcc-84b8-ea0460ead623 None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:48.304260 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58310e9a-c396-496a-ae1a-a503e96bad24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:48.305179 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58310e9a-c396-496a-ae1a-a503e96bad24 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:48.305561 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58310e9a-c396-496a-ae1a-a503e96bad24 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:48.305961 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58310e9a-c396-496a-ae1a-a503e96bad24 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:48.317738 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58310e9a-c396-496a-ae1a-a503e96bad24 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:48.319393 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58310e9a-c396-496a-ae1a-a503e96bad24 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:48.335405 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application [None req-58310e9a-c396-496a-ae1a-a503e96bad24 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 01:04:48.335405 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:04:48.335405 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:04:48.335405 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:04:48.335405 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:04:48.335405 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:04:48.335405 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:04:48.335405 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:04:48.335405 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:04:48.335405 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:04:48.335405 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:04:48.335405 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:04:48.335405 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 01:04:48.335405 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 01:04:48.335405 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 01:04:48.335405 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 01:04:48.337246 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:04:48.337246 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:04:48.337246 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:04:48.337246 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:04:48.337246 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:04:48.337246 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:04:48.337246 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:04:48.337246 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application with Lock( Aug 31 01:04:48.337246 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:04:48.337246 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:04:48.337246 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:04:48.337246 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:04:48.337246 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:04:48.337246 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:04:48.337246 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:04:48.337246 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:04:48.337246 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 01:04:48.339410 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 01:04:48.339410 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 01:04:48.339410 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Aug 31 01:04:48.344748 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 410/794] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 01:04:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:04:48.344748 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58310e9a-c396-496a-ae1a-a503e96bad24 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:48.345179 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 385/795] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:04:48 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 01:04:48.349664 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79c94af-c122-4094-9306-256d367895c5 None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:48.440643 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:48.467557 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:48.476662 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58310e9a-c396-496a-ae1a-a503e96bad24 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:48.487302 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79c94af-c122-4094-9306-256d367895c5 None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:48.514073 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab1b692-fe4e-48d6-bdec-b48fe69e71cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:48.514711 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab1b692-fe4e-48d6-bdec-b48fe69e71cb None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:48.514995 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab1b692-fe4e-48d6-bdec-b48fe69e71cb None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:48.515273 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab1b692-fe4e-48d6-bdec-b48fe69e71cb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:48.526588 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab1b692-fe4e-48d6-bdec-b48fe69e71cb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:48.528267 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab1b692-fe4e-48d6-bdec-b48fe69e71cb None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:48.532423 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd28d1ed-8d37-4ea4-838a-5bc259cf6628 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:48.533437 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd28d1ed-8d37-4ea4-838a-5bc259cf6628 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:48.533924 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd28d1ed-8d37-4ea4-838a-5bc259cf6628 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:48.534422 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd28d1ed-8d37-4ea4-838a-5bc259cf6628 None admin] PATH_INFO: `/v3/domains` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:48.535353 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd28d1ed-8d37-4ea4-838a-5bc259cf6628 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:04:48.536501 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd28d1ed-8d37-4ea4-838a-5bc259cf6628 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:48.538688 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd28d1ed-8d37-4ea4-838a-5bc259cf6628 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:48.544883 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application [None req-3ab1b692-fe4e-48d6-bdec-b48fe69e71cb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 01:04:48.544883 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:04:48.544883 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:04:48.544883 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:04:48.544883 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:04:48.544883 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:04:48.544883 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:04:48.544883 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:04:48.544883 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:04:48.544883 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:04:48.544883 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:04:48.544883 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:04:48.544883 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 01:04:48.544883 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 01:04:48.544883 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 01:04:48.544883 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 01:04:48.545881 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:04:48.545881 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:04:48.545881 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:04:48.545881 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:04:48.545881 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:04:48.545881 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:04:48.545881 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:04:48.545881 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application with Lock( Aug 31 01:04:48.545881 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:04:48.545881 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:04:48.545881 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:04:48.545881 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:04:48.545881 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:04:48.545881 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:04:48.545881 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:04:48.545881 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:04:48.545881 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 01:04:48.547375 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 01:04:48.547375 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 01:04:48.547375 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Aug 31 01:04:48.547375 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 411/796] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 01:04:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:04:48.547724 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217fb281-647b-4c82-8ecd-68d23c62d8de None None] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:48.548013 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217fb281-647b-4c82-8ecd-68d23c62d8de None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:48.548293 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217fb281-647b-4c82-8ecd-68d23c62d8de None None] PATH_INFO: `/` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:48.553726 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 412/797] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:04:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:04:48.554594 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217fb281-647b-4c82-8ecd-68d23c62d8de None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:48.556222 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 386/798] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 31 01:04:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:04:48.557838 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c67a67-b903-455d-9d8e-de00c380fbb1 None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:48.558394 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c67a67-b903-455d-9d8e-de00c380fbb1 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:48.558899 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c67a67-b903-455d-9d8e-de00c380fbb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:48.581199 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-f6c67a67-b903-455d-9d8e-de00c380fbb1 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:04:48.583553 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-f6c67a67-b903-455d-9d8e-de00c380fbb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:04:48.666387 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 387/799] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:04:48 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 01:04:48.667217 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c67a67-b903-455d-9d8e-de00c380fbb1 None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:48.678701 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217fb281-647b-4c82-8ecd-68d23c62d8de None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:48.700118 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:48.714447 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fff23b-d048-44a0-875d-7e7c0edd139f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:48.715055 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fff23b-d048-44a0-875d-7e7c0edd139f None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:48.715414 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fff23b-d048-44a0-875d-7e7c0edd139f None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:48.715610 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fff23b-d048-44a0-875d-7e7c0edd139f None admin] PATH_INFO: `/v3/domains` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:48.716226 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fff23b-d048-44a0-875d-7e7c0edd139f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:04:48.716525 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fff23b-d048-44a0-875d-7e7c0edd139f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:48.717944 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fff23b-d048-44a0-875d-7e7c0edd139f None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:48.730521 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 413/800] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 31 01:04:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:04:48.731344 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fff23b-d048-44a0-875d-7e7c0edd139f None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:48.774879 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:48.808610 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c67a67-b903-455d-9d8e-de00c380fbb1 None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:48.840348 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fff23b-d048-44a0-875d-7e7c0edd139f None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:48.857366 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f023f16-3e86-4d30-8fba-ca2f75ba3eb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:48.858540 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f023f16-3e86-4d30-8fba-ca2f75ba3eb9 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:48.859353 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f023f16-3e86-4d30-8fba-ca2f75ba3eb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:48.860261 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f023f16-3e86-4d30-8fba-ca2f75ba3eb9 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:48.870571 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f023f16-3e86-4d30-8fba-ca2f75ba3eb9 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:48.873254 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f023f16-3e86-4d30-8fba-ca2f75ba3eb9 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:48.878912 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c26182-9e42-4188-b381-31820c323a59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:48.879545 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c26182-9e42-4188-b381-31820c323a59 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:48.879824 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c26182-9e42-4188-b381-31820c323a59 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:48.880095 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c26182-9e42-4188-b381-31820c323a59 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:48.890580 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application [None req-0f023f16-3e86-4d30-8fba-ca2f75ba3eb9 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 31 01:04:48.890580 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:04:48.890580 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:04:48.890580 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:04:48.890580 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:04:48.890580 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:04:48.890580 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:04:48.890580 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:04:48.890580 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:04:48.890580 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:04:48.890580 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:04:48.890580 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:04:48.890580 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 01:04:48.890580 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 01:04:48.890580 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 01:04:48.890580 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 01:04:48.891543 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:04:48.891543 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:04:48.891543 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 01:04:48.891543 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 01:04:48.891543 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 01:04:48.891543 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 01:04:48.891543 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:04:48.891543 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:04:48.891543 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:04:48.891543 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:04:48.891543 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:04:48.891543 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application with Lock( Aug 31 01:04:48.891543 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:04:48.891543 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:04:48.891543 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:04:48.891543 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:04:48.891543 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:04:48.891543 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:04:48.892580 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:04:48.892580 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:04:48.892580 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 01:04:48.892580 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 01:04:48.892580 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 01:04:48.892580 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 01:04:48.892580 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 01:04:48.892580 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 01:04:48.892580 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 31 01:04:48.892580 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Aug 31 01:04:48.893142 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c26182-9e42-4188-b381-31820c323a59 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 01:04:48.895603 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 388/801] 10.222.0.20 () {56 vars in 1093 bytes} [Wed Aug 31 01:04:48 2022] GET /identity/v3/users/nova => generated 82 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:04:48.896757 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f023f16-3e86-4d30-8fba-ca2f75ba3eb9 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:48.906187 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application [None req-03c26182-9e42-4188-b381-31820c323a59 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 01:04:48.906187 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:04:48.906187 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:04:48.906187 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:04:48.906187 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:04:48.906187 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:04:48.906187 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:04:48.906187 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:04:48.906187 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:04:48.906187 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:04:48.906187 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:04:48.906187 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:04:48.906187 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 01:04:48.906187 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 01:04:48.906187 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 01:04:48.906187 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 01:04:48.907098 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 01:04:48.907098 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 01:04:48.907098 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 01:04:48.907098 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 01:04:48.907098 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:04:48.907098 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:04:48.907098 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:04:48.907098 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:04:48.907098 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:04:48.907098 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:04:48.907098 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:04:48.907098 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application with Lock( Aug 31 01:04:48.907098 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:04:48.907098 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:04:48.907098 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:04:48.907098 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:04:48.908519 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:04:48.908519 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:04:48.908519 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:04:48.908519 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:04:48.908519 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 01:04:48.908519 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 01:04:48.908519 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 01:04:48.908519 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 01:04:48.908519 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 01:04:48.908519 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 01:04:48.908519 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 01:04:48.908519 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Aug 31 01:04:48.908519 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 414/802] 10.222.0.20 () {56 vars in 1102 bytes} [Wed Aug 31 01:04:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:04:48.909258 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c26182-9e42-4188-b381-31820c323a59 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:48.958249 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:49.019737 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c26182-9e42-4188-b381-31820c323a59 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:49.019969 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f023f16-3e86-4d30-8fba-ca2f75ba3eb9 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:49.054803 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6afa66-4958-4356-9ca6-c12bd4030167 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:49.055407 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6afa66-4958-4356-9ca6-c12bd4030167 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:49.055689 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6afa66-4958-4356-9ca6-c12bd4030167 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:49.055959 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6afa66-4958-4356-9ca6-c12bd4030167 None admin] PATH_INFO: `/v3/users` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:49.056301 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a0d8a1-7f9b-48f3-970b-7f0e84c41ae5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:49.056685 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6afa66-4958-4356-9ca6-c12bd4030167 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:04:49.056964 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a0d8a1-7f9b-48f3-970b-7f0e84c41ae5 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:49.057064 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6afa66-4958-4356-9ca6-c12bd4030167 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:49.057249 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a0d8a1-7f9b-48f3-970b-7f0e84c41ae5 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:49.057524 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a0d8a1-7f9b-48f3-970b-7f0e84c41ae5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:49.058435 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6afa66-4958-4356-9ca6-c12bd4030167 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:49.060330 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:49.065582 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a0d8a1-7f9b-48f3-970b-7f0e84c41ae5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:49.067284 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a0d8a1-7f9b-48f3-970b-7f0e84c41ae5 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:49.075879 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 415/803] 10.222.0.20 () {56 vars in 1133 bytes} [Wed Aug 31 01:04:48 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:04:49.077155 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6afa66-4958-4356-9ca6-c12bd4030167 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:49.079096 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application [None req-15a0d8a1-7f9b-48f3-970b-7f0e84c41ae5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 01:04:49.079096 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:04:49.079096 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:04:49.079096 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:04:49.079096 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:04:49.079096 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:04:49.079096 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:04:49.079096 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:04:49.079096 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:04:49.079096 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:04:49.079096 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:04:49.079096 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:04:49.079096 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 01:04:49.079096 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 01:04:49.079096 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 01:04:49.079963 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 01:04:49.079963 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:04:49.079963 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:04:49.079963 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:04:49.079963 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:04:49.079963 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:04:49.079963 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:04:49.079963 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:04:49.079963 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application with Lock( Aug 31 01:04:49.079963 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:04:49.079963 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:04:49.079963 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:04:49.079963 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:04:49.079963 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:04:49.079963 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:04:49.079963 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:04:49.079963 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:04:49.081430 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 01:04:49.081430 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 01:04:49.081430 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 01:04:49.081430 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 01:04:49.081430 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 01:04:49.081430 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 01:04:49.081430 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 01:04:49.081430 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Aug 31 01:04:49.081430 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 389/804] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Aug 31 01:04:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:04:49.081430 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a0d8a1-7f9b-48f3-970b-7f0e84c41ae5 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:49.192141 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a0d8a1-7f9b-48f3-970b-7f0e84c41ae5 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:49.195812 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:49.200019 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6afa66-4958-4356-9ca6-c12bd4030167 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:49.230575 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedd0023-980b-4f5a-99b7-da2f75a4cb16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:49.231434 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedd0023-980b-4f5a-99b7-da2f75a4cb16 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:49.231969 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedd0023-980b-4f5a-99b7-da2f75a4cb16 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:49.232256 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedd0023-980b-4f5a-99b7-da2f75a4cb16 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:49.232891 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedd0023-980b-4f5a-99b7-da2f75a4cb16 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:49.234265 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedd0023-980b-4f5a-99b7-da2f75a4cb16 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:49.235712 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add86e1a-1c9d-4fa0-91ee-5fb6820a7223 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:49.236319 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add86e1a-1c9d-4fa0-91ee-5fb6820a7223 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:49.236611 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add86e1a-1c9d-4fa0-91ee-5fb6820a7223 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:49.236903 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add86e1a-1c9d-4fa0-91ee-5fb6820a7223 None admin] PATH_INFO: `/v3/roles` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:49.237560 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add86e1a-1c9d-4fa0-91ee-5fb6820a7223 None admin] RBAC: Adding query filter params (name=service) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:04:49.237945 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add86e1a-1c9d-4fa0-91ee-5fb6820a7223 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:49.239210 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add86e1a-1c9d-4fa0-91ee-5fb6820a7223 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:49.243983 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application [None req-bedd0023-980b-4f5a-99b7-da2f75a4cb16 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 31 01:04:49.243983 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:04:49.243983 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 01:04:49.243983 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 01:04:49.243983 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 01:04:49.243983 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 01:04:49.243983 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 01:04:49.243983 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 01:04:49.243983 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 31 01:04:49.243983 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Aug 31 01:04:49.243983 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 01:04:49.243983 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Aug 31 01:04:49.243983 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:04:49.243983 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:04:49.243983 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:04:49.243983 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:04:49.243983 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:04:49.245172 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:04:49.245172 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:04:49.245172 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:04:49.245172 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:04:49.245172 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:04:49.245172 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:04:49.245172 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 01:04:49.245172 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 01:04:49.245172 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 01:04:49.245172 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 01:04:49.245172 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:04:49.245172 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:04:49.245172 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:04:49.245172 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:04:49.245172 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:04:49.245172 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:04:49.248799 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:04:49.248799 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application with Lock( Aug 31 01:04:49.248799 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:04:49.248799 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:04:49.248799 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:04:49.248799 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:04:49.248799 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:04:49.248799 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:04:49.248799 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:04:49.248799 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:04:49.248799 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 01:04:49.248799 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 01:04:49.248799 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 31 01:04:49.248799 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Aug 31 01:04:49.248799 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 390/805] 10.222.0.20 () {56 vars in 1129 bytes} [Wed Aug 31 01:04:49 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:04:49.249661 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedd0023-980b-4f5a-99b7-da2f75a4cb16 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:49.249739 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 416/806] 10.222.0.20 () {56 vars in 1103 bytes} [Wed Aug 31 01:04:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:04:49.249739 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add86e1a-1c9d-4fa0-91ee-5fb6820a7223 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:49.315916 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:49.357099 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add86e1a-1c9d-4fa0-91ee-5fb6820a7223 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:49.361675 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedd0023-980b-4f5a-99b7-da2f75a4cb16 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:49.393174 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f64b7e-0cb2-4d75-adc6-9b33bef13aca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:49.393799 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f64b7e-0cb2-4d75-adc6-9b33bef13aca None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:49.394080 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f64b7e-0cb2-4d75-adc6-9b33bef13aca None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:49.394357 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f64b7e-0cb2-4d75-adc6-9b33bef13aca None admin] PATH_INFO: `/v3/projects` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:49.395063 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f64b7e-0cb2-4d75-adc6-9b33bef13aca None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:04:49.395371 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f64b7e-0cb2-4d75-adc6-9b33bef13aca None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:49.396824 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f64b7e-0cb2-4d75-adc6-9b33bef13aca None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:49.399608 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6901082e-6118-4332-bbf9-625aa4393f6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:49.400213 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6901082e-6118-4332-bbf9-625aa4393f6d None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:49.400499 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6901082e-6118-4332-bbf9-625aa4393f6d None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:49.400810 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6901082e-6118-4332-bbf9-625aa4393f6d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:49.409384 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 417/807] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Aug 31 01:04:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:04:49.410218 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f64b7e-0cb2-4d75-adc6-9b33bef13aca None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:49.430426 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:49.469795 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6901082e-6118-4332-bbf9-625aa4393f6d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:49.471257 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6901082e-6118-4332-bbf9-625aa4393f6d None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:49.507836 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f64b7e-0cb2-4d75-adc6-9b33bef13aca None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:49.527625 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:49.540415 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90737df5-b68c-4dda-ab44-0397089ba0ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:49.541057 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90737df5-b68c-4dda-ab44-0397089ba0ac None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:49.541339 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90737df5-b68c-4dda-ab44-0397089ba0ac None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:49.541610 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90737df5-b68c-4dda-ab44-0397089ba0ac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:49.551700 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90737df5-b68c-4dda-ab44-0397089ba0ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:49.553200 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90737df5-b68c-4dda-ab44-0397089ba0ac None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:49.567660 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 391/808] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Aug 31 01:04:49 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 01:04:49.568131 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application [None req-90737df5-b68c-4dda-ab44-0397089ba0ac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 01:04:49.568131 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:04:49.568131 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:04:49.568131 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:04:49.568131 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:04:49.568131 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:04:49.568131 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:04:49.568131 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:04:49.568131 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:04:49.568131 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:04:49.568131 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:04:49.568131 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:04:49.568131 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 01:04:49.568131 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 01:04:49.568131 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 01:04:49.568131 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 01:04:49.569091 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:04:49.569091 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:04:49.569091 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:04:49.569091 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:04:49.569091 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:04:49.569091 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:04:49.569091 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:04:49.569091 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application with Lock( Aug 31 01:04:49.569091 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:04:49.569091 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:04:49.569091 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:04:49.569091 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:04:49.569091 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:04:49.569091 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:04:49.569091 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:04:49.569091 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:04:49.569091 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 01:04:49.570617 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 01:04:49.570617 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 01:04:49.570617 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Aug 31 01:04:49.570808 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6901082e-6118-4332-bbf9-625aa4393f6d None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:49.570879 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 418/809] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 01:04:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:04:49.571428 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0484f913-d730-42c7-a051-af586d50ec23 None None] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:49.571715 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0484f913-d730-42c7-a051-af586d50ec23 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:49.572000 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0484f913-d730-42c7-a051-af586d50ec23 None None] PATH_INFO: `/` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:49.573509 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 419/810] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:04:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:04:49.574294 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0484f913-d730-42c7-a051-af586d50ec23 None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:49.650313 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:49.668799 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6901082e-6118-4332-bbf9-625aa4393f6d None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:49.679930 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0484f913-d730-42c7-a051-af586d50ec23 None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:49.703765 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825e45ca-c6a0-4145-b15a-a8d31fcd46c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:49.704376 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825e45ca-c6a0-4145-b15a-a8d31fcd46c1 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:49.704662 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825e45ca-c6a0-4145-b15a-a8d31fcd46c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:49.704969 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825e45ca-c6a0-4145-b15a-a8d31fcd46c1 None admin] PATH_INFO: `/v3/services` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:49.705601 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825e45ca-c6a0-4145-b15a-a8d31fcd46c1 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:04:49.705911 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825e45ca-c6a0-4145-b15a-a8d31fcd46c1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:49.707280 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825e45ca-c6a0-4145-b15a-a8d31fcd46c1 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:49.711862 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 392/811] 10.222.0.20 () {56 vars in 1124 bytes} [Wed Aug 31 01:04:49 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:04:49.712980 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822f369d-a0fc-4496-abd6-c6ae92e3c53c None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:49.713269 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822f369d-a0fc-4496-abd6-c6ae92e3c53c None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:49.713547 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822f369d-a0fc-4496-abd6-c6ae92e3c53c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:49.718232 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1050ba-05c6-43b6-bcad-148fc6185055 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:49.718895 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1050ba-05c6-43b6-bcad-148fc6185055 None admin] REQUEST_METHOD: `PUT` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:49.719180 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1050ba-05c6-43b6-bcad-148fc6185055 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:49.719461 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1050ba-05c6-43b6-bcad-148fc6185055 None admin] PATH_INFO: `/v3/projects/60b2abc15c2849ba8a1ed92117cc981b/users/fc1d010919ec40878ae829cc8dceec4c/roles/d009411496b8452b88e77ffb8db015a7` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:49.721423 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1050ba-05c6-43b6-bcad-148fc6185055 None admin] RBAC: Authorizing `identity:create_grant(project_id=60b2abc15c2849ba8a1ed92117cc981b, user_id=fc1d010919ec40878ae829cc8dceec4c, role_id=d009411496b8452b88e77ffb8db015a7)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:49.722845 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1050ba-05c6-43b6-bcad-148fc6185055 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:49.730942 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-822f369d-a0fc-4496-abd6-c6ae92e3c53c None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:04:49.732132 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-822f369d-a0fc-4496-abd6-c6ae92e3c53c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:04:49.749610 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:49.797210 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 393/812] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:04:49 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 01:04:49.797957 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822f369d-a0fc-4496-abd6-c6ae92e3c53c None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:49.814581 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 420/813] 10.222.0.20 () {58 vars in 1439 bytes} [Wed Aug 31 01:04:49 2022] PUT /identity/v3/projects/60b2abc15c2849ba8a1ed92117cc981b/users/fc1d010919ec40878ae829cc8dceec4c/roles/d009411496b8452b88e77ffb8db015a7 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:04:49.814581 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1050ba-05c6-43b6-bcad-148fc6185055 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:49.942839 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822f369d-a0fc-4496-abd6-c6ae92e3c53c None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:49.956524 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1050ba-05c6-43b6-bcad-148fc6185055 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:49.994072 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e511f4fd-736c-4f74-ba56-984630386760 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:49.994746 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e511f4fd-736c-4f74-ba56-984630386760 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:49.995030 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e511f4fd-736c-4f74-ba56-984630386760 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:49.995307 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e511f4fd-736c-4f74-ba56-984630386760 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:49.995528 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:50.002763 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11df66a4-181a-4543-9d6c-95f453fdbb28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:50.003614 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11df66a4-181a-4543-9d6c-95f453fdbb28 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:50.008853 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11df66a4-181a-4543-9d6c-95f453fdbb28 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:50.008853 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11df66a4-181a-4543-9d6c-95f453fdbb28 None admin] PATH_INFO: `/v3/domains` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:50.008853 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11df66a4-181a-4543-9d6c-95f453fdbb28 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:04:50.008853 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11df66a4-181a-4543-9d6c-95f453fdbb28 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:50.009159 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e511f4fd-736c-4f74-ba56-984630386760 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:50.009159 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e511f4fd-736c-4f74-ba56-984630386760 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:50.013786 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11df66a4-181a-4543-9d6c-95f453fdbb28 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:50.024549 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application [None req-e511f4fd-736c-4f74-ba56-984630386760 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 01:04:50.024549 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:04:50.024549 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:04:50.024549 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:04:50.024549 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:04:50.024549 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:04:50.024549 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:04:50.024549 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:04:50.024549 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:04:50.024549 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:04:50.024549 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:04:50.024549 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:04:50.024549 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 01:04:50.024549 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 01:04:50.024549 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 01:04:50.024549 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 01:04:50.025671 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:04:50.025671 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:04:50.025671 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:04:50.025671 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:04:50.025671 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:04:50.025671 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:04:50.025671 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:04:50.025671 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application with Lock( Aug 31 01:04:50.025671 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:04:50.025671 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:04:50.025671 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:04:50.025671 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:04:50.025671 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:04:50.025671 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:04:50.025671 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:04:50.025671 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:04:50.025671 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 01:04:50.026541 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 01:04:50.026541 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 01:04:50.026541 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Aug 31 01:04:50.027289 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 394/814] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 01:04:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:04:50.028679 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e511f4fd-736c-4f74-ba56-984630386760 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:50.034855 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 421/815] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 31 01:04:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:04:50.036017 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11df66a4-181a-4543-9d6c-95f453fdbb28 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:50.039799 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:50.142061 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e511f4fd-736c-4f74-ba56-984630386760 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:50.148868 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11df66a4-181a-4543-9d6c-95f453fdbb28 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:50.178374 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4907a50a-56c5-4597-b536-fef0e5b70e70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:50.179814 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4907a50a-56c5-4597-b536-fef0e5b70e70 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:50.180283 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4907a50a-56c5-4597-b536-fef0e5b70e70 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:50.180555 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4907a50a-56c5-4597-b536-fef0e5b70e70 None admin] PATH_INFO: `/v3/services` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:50.181212 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4907a50a-56c5-4597-b536-fef0e5b70e70 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:04:50.181513 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4907a50a-56c5-4597-b536-fef0e5b70e70 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:50.183064 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4907a50a-56c5-4597-b536-fef0e5b70e70 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:50.185237 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5754a71-bad5-453d-8aba-f7cd05f6a9e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:50.185845 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5754a71-bad5-453d-8aba-f7cd05f6a9e8 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:50.186128 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5754a71-bad5-453d-8aba-f7cd05f6a9e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:50.186401 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5754a71-bad5-453d-8aba-f7cd05f6a9e8 None admin] PATH_INFO: `/v3/domains` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:50.187041 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5754a71-bad5-453d-8aba-f7cd05f6a9e8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:04:50.187338 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5754a71-bad5-453d-8aba-f7cd05f6a9e8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:50.188684 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5754a71-bad5-453d-8aba-f7cd05f6a9e8 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:50.189734 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 395/816] 10.222.0.20 () {56 vars in 1124 bytes} [Wed Aug 31 01:04:50 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:04:50.190757 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4907a50a-56c5-4597-b536-fef0e5b70e70 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:50.204601 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 422/817] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 31 01:04:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:04:50.205678 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5754a71-bad5-453d-8aba-f7cd05f6a9e8 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:50.261122 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:50.270427 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:50.305082 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4907a50a-56c5-4597-b536-fef0e5b70e70 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:50.311385 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5754a71-bad5-453d-8aba-f7cd05f6a9e8 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:50.339643 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569e2cfc-98d1-4d02-acef-0e30a1fb152f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:50.340617 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569e2cfc-98d1-4d02-acef-0e30a1fb152f None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:50.341135 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569e2cfc-98d1-4d02-acef-0e30a1fb152f None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:50.341416 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569e2cfc-98d1-4d02-acef-0e30a1fb152f None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:50.348341 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69ca5a5-7c5e-4641-9875-25f8e7731cd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:50.348997 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69ca5a5-7c5e-4641-9875-25f8e7731cd1 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:50.349270 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569e2cfc-98d1-4d02-acef-0e30a1fb152f None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:50.349395 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69ca5a5-7c5e-4641-9875-25f8e7731cd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:50.349555 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69ca5a5-7c5e-4641-9875-25f8e7731cd1 None admin] PATH_INFO: `/v3/services/network` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:50.350104 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69ca5a5-7c5e-4641-9875-25f8e7731cd1 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:50.351378 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569e2cfc-98d1-4d02-acef-0e30a1fb152f None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:50.352085 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69ca5a5-7c5e-4641-9875-25f8e7731cd1 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:50.360923 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application [None req-d69ca5a5-7c5e-4641-9875-25f8e7731cd1 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 31 01:04:50.360923 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:04:50.360923 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 01:04:50.360923 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 01:04:50.360923 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 01:04:50.360923 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 01:04:50.360923 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 01:04:50.360923 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 01:04:50.360923 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 31 01:04:50.360923 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Aug 31 01:04:50.360923 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 01:04:50.360923 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Aug 31 01:04:50.360923 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:04:50.360923 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:04:50.360923 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:04:50.360923 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:04:50.360923 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:04:50.361894 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:04:50.361894 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:04:50.361894 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:04:50.361894 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:04:50.361894 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:04:50.361894 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:04:50.361894 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 01:04:50.361894 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 01:04:50.361894 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 01:04:50.361894 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 01:04:50.361894 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:04:50.361894 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:04:50.361894 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:04:50.361894 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:04:50.361894 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:04:50.361894 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:04:50.362750 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:04:50.362750 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application with Lock( Aug 31 01:04:50.362750 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:04:50.362750 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:04:50.362750 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:04:50.362750 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:04:50.362750 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:04:50.362750 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:04:50.362750 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:04:50.362750 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:04:50.362750 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 01:04:50.362750 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 01:04:50.362750 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 31 01:04:50.362750 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Aug 31 01:04:50.362750 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 423/818] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Aug 31 01:04:50 2022] GET /identity/v3/services/network => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:04:50.363531 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69ca5a5-7c5e-4641-9875-25f8e7731cd1 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:50.367558 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application [None req-569e2cfc-98d1-4d02-acef-0e30a1fb152f None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 31 01:04:50.367558 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:04:50.367558 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:04:50.367558 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:04:50.367558 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:04:50.367558 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:04:50.367558 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:04:50.367558 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:04:50.367558 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:04:50.367558 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:04:50.367558 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:04:50.367558 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:04:50.367558 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 01:04:50.367558 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 01:04:50.367558 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 01:04:50.367558 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 01:04:50.368479 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:04:50.368479 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:04:50.368479 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 01:04:50.368479 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 01:04:50.368479 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 01:04:50.368479 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 01:04:50.368479 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:04:50.368479 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:04:50.368479 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:04:50.368479 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:04:50.368479 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:04:50.368479 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application with Lock( Aug 31 01:04:50.368479 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:04:50.368479 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:04:50.368479 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:04:50.368479 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:04:50.368479 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:04:50.368479 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:04:50.369451 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:04:50.369451 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:04:50.369451 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 01:04:50.369451 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 01:04:50.369451 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 01:04:50.369451 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 01:04:50.369451 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 01:04:50.369451 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 01:04:50.369451 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 31 01:04:50.369451 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Aug 31 01:04:50.369451 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 396/819] 10.222.0.20 () {56 vars in 1117 bytes} [Wed Aug 31 01:04:50 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:04:50.370527 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569e2cfc-98d1-4d02-acef-0e30a1fb152f None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:50.471792 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69ca5a5-7c5e-4641-9875-25f8e7731cd1 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:50.478998 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569e2cfc-98d1-4d02-acef-0e30a1fb152f None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:50.489211 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:50.506227 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc0dfc6-38ab-4eb8-895b-870bffb12c81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:50.506855 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc0dfc6-38ab-4eb8-895b-870bffb12c81 None admin] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:50.507137 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc0dfc6-38ab-4eb8-895b-870bffb12c81 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:50.507424 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc0dfc6-38ab-4eb8-895b-870bffb12c81 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:50.508080 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc0dfc6-38ab-4eb8-895b-870bffb12c81 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:50.509295 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:50.509476 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc0dfc6-38ab-4eb8-895b-870bffb12c81 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:50.515279 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55954a09-96c1-41cc-b85c-5c7cbeab4adf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:50.515899 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55954a09-96c1-41cc-b85c-5c7cbeab4adf None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:50.516195 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55954a09-96c1-41cc-b85c-5c7cbeab4adf None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:50.516455 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55954a09-96c1-41cc-b85c-5c7cbeab4adf None admin] PATH_INFO: `/v3/projects/service` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:50.523621 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55954a09-96c1-41cc-b85c-5c7cbeab4adf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:50.525077 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55954a09-96c1-41cc-b85c-5c7cbeab4adf None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:50.536257 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application [None req-55954a09-96c1-41cc-b85c-5c7cbeab4adf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 01:04:50.536257 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:04:50.536257 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:04:50.536257 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:04:50.536257 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:04:50.536257 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:04:50.536257 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:04:50.536257 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:04:50.536257 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:04:50.536257 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:04:50.536257 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:04:50.536257 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:04:50.536257 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 01:04:50.536257 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 01:04:50.536257 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 01:04:50.537341 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 01:04:50.537341 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:04:50.537341 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:04:50.537341 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:04:50.537341 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:04:50.537341 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:04:50.537341 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:04:50.537341 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:04:50.537341 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application with Lock( Aug 31 01:04:50.537341 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:04:50.537341 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:04:50.537341 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:04:50.537341 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:04:50.537341 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:04:50.537341 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:04:50.537341 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:04:50.537341 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:04:50.543681 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 01:04:50.543681 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 01:04:50.543681 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 01:04:50.543681 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 01:04:50.543681 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 01:04:50.543681 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 01:04:50.543681 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 01:04:50.543681 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Aug 31 01:04:50.543681 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 397/820] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Aug 31 01:04:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:04:50.543681 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55954a09-96c1-41cc-b85c-5c7cbeab4adf None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:50.616754 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 424/821] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 01:04:50 2022] POST /identity/v3/endpoints => generated 346 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:04:50.616754 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc0dfc6-38ab-4eb8-895b-870bffb12c81 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:50.659483 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55954a09-96c1-41cc-b85c-5c7cbeab4adf None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:50.713469 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38870711-0a3a-497c-9710-5f5f3f3036ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:50.714343 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38870711-0a3a-497c-9710-5f5f3f3036ca None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:50.714635 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38870711-0a3a-497c-9710-5f5f3f3036ca None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:50.714908 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38870711-0a3a-497c-9710-5f5f3f3036ca None admin] PATH_INFO: `/v3/services` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:50.716000 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38870711-0a3a-497c-9710-5f5f3f3036ca None admin] RBAC: Adding query filter params (name=network) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:04:50.716309 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38870711-0a3a-497c-9710-5f5f3f3036ca None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:50.717892 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38870711-0a3a-497c-9710-5f5f3f3036ca None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:50.725350 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 398/822] 10.222.0.20 () {56 vars in 1112 bytes} [Wed Aug 31 01:04:50 2022] GET /identity/v3/services?name=network => generated 124 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:04:50.726282 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38870711-0a3a-497c-9710-5f5f3f3036ca None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:50.737847 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc0dfc6-38ab-4eb8-895b-870bffb12c81 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:50.788049 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8682cc25-3655-485c-96be-a7bb95d50a5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:50.788807 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8682cc25-3655-485c-96be-a7bb95d50a5e None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:50.789162 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8682cc25-3655-485c-96be-a7bb95d50a5e None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:50.790195 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8682cc25-3655-485c-96be-a7bb95d50a5e None admin] PATH_INFO: `/v3/users` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:50.790943 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8682cc25-3655-485c-96be-a7bb95d50a5e None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:04:50.791251 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8682cc25-3655-485c-96be-a7bb95d50a5e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:50.792693 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8682cc25-3655-485c-96be-a7bb95d50a5e None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:50.804835 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:50.814720 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 425/823] 10.222.0.20 () {56 vars in 1149 bytes} [Wed Aug 31 01:04:50 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:04:50.815571 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8682cc25-3655-485c-96be-a7bb95d50a5e None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:50.856392 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:50.866769 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38870711-0a3a-497c-9710-5f5f3f3036ca None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:50.913057 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fb147c-8f7b-4f33-9882-e3dbe2d9dc51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:50.913853 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fb147c-8f7b-4f33-9882-e3dbe2d9dc51 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:50.914135 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fb147c-8f7b-4f33-9882-e3dbe2d9dc51 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:50.914466 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fb147c-8f7b-4f33-9882-e3dbe2d9dc51 None admin] PATH_INFO: `/v3/projects` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:50.915555 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fb147c-8f7b-4f33-9882-e3dbe2d9dc51 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:04:50.915870 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fb147c-8f7b-4f33-9882-e3dbe2d9dc51 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:50.917951 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fb147c-8f7b-4f33-9882-e3dbe2d9dc51 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:50.936005 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 399/824] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Aug 31 01:04:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:04:50.936005 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fb147c-8f7b-4f33-9882-e3dbe2d9dc51 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:50.968274 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8682cc25-3655-485c-96be-a7bb95d50a5e None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:51.003477 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521e343e-97ec-47d0-9cc6-2056268c4742 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:51.004197 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521e343e-97ec-47d0-9cc6-2056268c4742 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:51.004572 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521e343e-97ec-47d0-9cc6-2056268c4742 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:51.005018 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521e343e-97ec-47d0-9cc6-2056268c4742 None admin] PATH_INFO: `/v3/services` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:51.005684 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521e343e-97ec-47d0-9cc6-2056268c4742 None admin] RBAC: Adding query filter params (type=network) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:04:51.006346 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521e343e-97ec-47d0-9cc6-2056268c4742 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:51.007601 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521e343e-97ec-47d0-9cc6-2056268c4742 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:51.013920 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 426/825] 10.222.0.20 () {56 vars in 1112 bytes} [Wed Aug 31 01:04:50 2022] GET /identity/v3/services?type=network => generated 350 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:04:51.014866 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521e343e-97ec-47d0-9cc6-2056268c4742 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:51.061477 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fb147c-8f7b-4f33-9882-e3dbe2d9dc51 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:51.077031 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:51.106392 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a18f595-b68c-43ca-a7db-ead034135317 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:51.107585 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a18f595-b68c-43ca-a7db-ead034135317 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:51.108048 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a18f595-b68c-43ca-a7db-ead034135317 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:51.108471 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a18f595-b68c-43ca-a7db-ead034135317 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:51.116999 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:51.122574 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a18f595-b68c-43ca-a7db-ead034135317 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:51.124301 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a18f595-b68c-43ca-a7db-ead034135317 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:51.125934 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521e343e-97ec-47d0-9cc6-2056268c4742 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:51.146919 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application [None req-8a18f595-b68c-43ca-a7db-ead034135317 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 01:04:51.146919 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:04:51.146919 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:04:51.146919 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:04:51.146919 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:04:51.146919 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:04:51.146919 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:04:51.146919 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:04:51.146919 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:04:51.146919 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:04:51.146919 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:04:51.146919 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:04:51.146919 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 01:04:51.146919 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 01:04:51.146919 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 01:04:51.146919 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 01:04:51.147928 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:04:51.147928 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:04:51.147928 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:04:51.147928 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:04:51.147928 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:04:51.147928 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:04:51.147928 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:04:51.147928 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application with Lock( Aug 31 01:04:51.147928 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:04:51.147928 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:04:51.147928 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:04:51.147928 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:04:51.147928 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:04:51.147928 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:04:51.147928 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:04:51.147928 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:04:51.147928 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 01:04:51.149548 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 01:04:51.149548 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 01:04:51.149548 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Aug 31 01:04:51.152968 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 400/826] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 01:04:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:04:51.153708 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a18f595-b68c-43ca-a7db-ead034135317 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:51.169541 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174d698c-bb77-4222-a1d2-211a3175020a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:51.170538 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174d698c-bb77-4222-a1d2-211a3175020a None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:51.171046 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174d698c-bb77-4222-a1d2-211a3175020a None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:51.171318 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174d698c-bb77-4222-a1d2-211a3175020a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:51.172042 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174d698c-bb77-4222-a1d2-211a3175020a None admin] RBAC: Adding query filter params (role.id=d009411496b8452b88e77ffb8db015a7, scope.project.id=60b2abc15c2849ba8a1ed92117cc981b, user.id=8744c9de803048f583d0317d01f6a540) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:04:51.172346 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174d698c-bb77-4222-a1d2-211a3175020a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:51.173907 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174d698c-bb77-4222-a1d2-211a3175020a None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:51.181036 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 427/827] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Aug 31 01:04:51 2022] GET /identity/v3/role_assignments?user.id=8744c9de803048f583d0317d01f6a540&scope.project.id=60b2abc15c2849ba8a1ed92117cc981b&role.id=d009411496b8452b88e77ffb8db015a7 => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:04:51.181947 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174d698c-bb77-4222-a1d2-211a3175020a None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:51.303174 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a18f595-b68c-43ca-a7db-ead034135317 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:51.345362 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6a8bc2-b0b0-4ed6-a730-252c64d233a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:51.345996 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6a8bc2-b0b0-4ed6-a730-252c64d233a7 None admin] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:51.346336 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6a8bc2-b0b0-4ed6-a730-252c64d233a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:51.346615 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6a8bc2-b0b0-4ed6-a730-252c64d233a7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:51.347302 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6a8bc2-b0b0-4ed6-a730-252c64d233a7 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:51.348701 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6a8bc2-b0b0-4ed6-a730-252c64d233a7 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:51.391930 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174d698c-bb77-4222-a1d2-211a3175020a None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:51.427533 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b033017-6426-43c2-a251-00976bfe4e0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:51.428146 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b033017-6426-43c2-a251-00976bfe4e0b None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:51.428430 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b033017-6426-43c2-a251-00976bfe4e0b None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:51.429316 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b033017-6426-43c2-a251-00976bfe4e0b None admin] PATH_INFO: `/v3/domains` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:51.429955 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b033017-6426-43c2-a251-00976bfe4e0b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:04:51.430260 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b033017-6426-43c2-a251-00976bfe4e0b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:51.431630 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b033017-6426-43c2-a251-00976bfe4e0b None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:51.444506 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 428/828] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 31 01:04:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:04:51.449343 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b033017-6426-43c2-a251-00976bfe4e0b None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:51.450866 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 401/829] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 01:04:51 2022] POST /identity/v3/endpoints => generated 337 bytes in 297 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:04:51.476581 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:51.594159 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b033017-6426-43c2-a251-00976bfe4e0b None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:51.649132 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91750500-4c5b-40c7-bb29-988627970c57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:51.649828 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91750500-4c5b-40c7-bb29-988627970c57 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:51.650117 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91750500-4c5b-40c7-bb29-988627970c57 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:51.650393 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91750500-4c5b-40c7-bb29-988627970c57 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:51.659067 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91750500-4c5b-40c7-bb29-988627970c57 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:51.660594 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91750500-4c5b-40c7-bb29-988627970c57 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:51.675157 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application [None req-91750500-4c5b-40c7-bb29-988627970c57 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 01:04:51.675157 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:04:51.675157 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:04:51.675157 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:04:51.675157 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:04:51.675157 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:04:51.675157 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:04:51.675157 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:04:51.675157 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:04:51.675157 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:04:51.675157 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:04:51.675157 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:04:51.675157 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 01:04:51.675157 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 01:04:51.675157 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 01:04:51.676243 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 01:04:51.676243 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:04:51.676243 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:04:51.676243 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:04:51.676243 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:04:51.676243 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:04:51.676243 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:04:51.676243 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:04:51.676243 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application with Lock( Aug 31 01:04:51.676243 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:04:51.676243 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:04:51.676243 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:04:51.676243 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:04:51.676243 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:04:51.676243 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:04:51.676243 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:04:51.676243 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:04:51.678047 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 01:04:51.678047 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 01:04:51.678047 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 01:04:51.678047 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 01:04:51.678047 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 01:04:51.678047 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 01:04:51.678047 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 01:04:51.678047 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Aug 31 01:04:51.678047 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 429/830] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Aug 31 01:04:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 01:04:51.682679 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6a8bc2-b0b0-4ed6-a730-252c64d233a7 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:51.706551 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:51.801822 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6a8bc2-b0b0-4ed6-a730-252c64d233a7 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:51.842853 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da594a83-fbad-4cf9-b315-f0a25e50317d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:51.843858 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da594a83-fbad-4cf9-b315-f0a25e50317d None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:51.844408 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da594a83-fbad-4cf9-b315-f0a25e50317d None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:51.844968 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da594a83-fbad-4cf9-b315-f0a25e50317d None admin] PATH_INFO: `/v3/projects` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:51.845934 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da594a83-fbad-4cf9-b315-f0a25e50317d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:04:51.846438 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da594a83-fbad-4cf9-b315-f0a25e50317d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:51.848212 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da594a83-fbad-4cf9-b315-f0a25e50317d None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:51.859295 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e86667-eba8-406b-80c0-6c582a70fb05 None None] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:51.859584 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e86667-eba8-406b-80c0-6c582a70fb05 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:51.859914 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e86667-eba8-406b-80c0-6c582a70fb05 None None] PATH_INFO: `/` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:51.861478 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 430/831] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:04:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 01:04:51.864783 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 402/832] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Aug 31 01:04:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:04:51.868754 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87e5751-a208-49a9-9af0-d95f8de33805 None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:51.868754 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87e5751-a208-49a9-9af0-d95f8de33805 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:51.868754 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87e5751-a208-49a9-9af0-d95f8de33805 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:51.870756 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da594a83-fbad-4cf9-b315-f0a25e50317d None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:51.908402 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-e87e5751-a208-49a9-9af0-d95f8de33805 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:04:51.909859 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-e87e5751-a208-49a9-9af0-d95f8de33805 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:04:52.000271 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 431/833] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:04:51 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 01:04:52.098098 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:52.121887 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da594a83-fbad-4cf9-b315-f0a25e50317d None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:52.157601 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ddc969-9935-496c-b2b9-93c073942bba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:52.158266 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ddc969-9935-496c-b2b9-93c073942bba None admin] REQUEST_METHOD: `PUT` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:52.158548 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ddc969-9935-496c-b2b9-93c073942bba None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:52.158832 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ddc969-9935-496c-b2b9-93c073942bba None admin] PATH_INFO: `/v3/projects/60b2abc15c2849ba8a1ed92117cc981b/users/b3efb4f9daac4a0ca05a36dd361f17e5/roles/7d00a4cfa13744a69604ac2cf1a5816e` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:52.173642 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ddc969-9935-496c-b2b9-93c073942bba None admin] RBAC: Authorizing `identity:create_grant(project_id=60b2abc15c2849ba8a1ed92117cc981b, user_id=b3efb4f9daac4a0ca05a36dd361f17e5, role_id=7d00a4cfa13744a69604ac2cf1a5816e)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:52.175093 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ddc969-9935-496c-b2b9-93c073942bba None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:52.258999 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 403/834] 10.222.0.20 () {58 vars in 1439 bytes} [Wed Aug 31 01:04:51 2022] PUT /identity/v3/projects/60b2abc15c2849ba8a1ed92117cc981b/users/b3efb4f9daac4a0ca05a36dd361f17e5/roles/7d00a4cfa13744a69604ac2cf1a5816e => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 01:04:52.267381 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87e5751-a208-49a9-9af0-d95f8de33805 None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:52.275222 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:52.393672 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87e5751-a208-49a9-9af0-d95f8de33805 None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:52.431993 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b6ff4e-97c0-4049-9135-636b12a5ee24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:52.432636 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b6ff4e-97c0-4049-9135-636b12a5ee24 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:52.432957 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b6ff4e-97c0-4049-9135-636b12a5ee24 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:52.433232 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b6ff4e-97c0-4049-9135-636b12a5ee24 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:52.444821 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b6ff4e-97c0-4049-9135-636b12a5ee24 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 01:04:52.450064 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application [None req-e3b6ff4e-97c0-4049-9135-636b12a5ee24 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 01:04:52.450064 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:04:52.450064 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:04:52.450064 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:04:52.450064 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:04:52.450064 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:04:52.450064 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:04:52.450064 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:04:52.450064 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:04:52.450064 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:04:52.450064 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:04:52.450064 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:04:52.450064 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 01:04:52.450064 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 01:04:52.450064 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 01:04:52.450064 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 01:04:52.450978 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 01:04:52.450978 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 01:04:52.450978 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 01:04:52.450978 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 01:04:52.450978 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:04:52.450978 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:04:52.450978 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:04:52.450978 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:04:52.450978 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:04:52.450978 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:04:52.450978 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:04:52.450978 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application with Lock( Aug 31 01:04:52.450978 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:04:52.450978 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:04:52.450978 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:04:52.450978 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:04:52.451838 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:04:52.451838 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:04:52.451838 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:04:52.451838 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:04:52.451838 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 01:04:52.451838 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 01:04:52.451838 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 01:04:52.451838 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 01:04:52.451838 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 01:04:52.451838 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 01:04:52.451838 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 01:04:52.451838 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Aug 31 01:04:52.451838 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 432/835] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Aug 31 01:04:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:04:52.458538 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ddc969-9935-496c-b2b9-93c073942bba None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:52.568849 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ddc969-9935-496c-b2b9-93c073942bba None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:52.597119 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:52.605777 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65ffcb6-1fd6-4feb-acb5-ea398ff1d2fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:52.606390 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65ffcb6-1fd6-4feb-acb5-ea398ff1d2fc None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:52.606670 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65ffcb6-1fd6-4feb-acb5-ea398ff1d2fc None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:52.607018 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65ffcb6-1fd6-4feb-acb5-ea398ff1d2fc None admin] PATH_INFO: `/v3/roles` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:52.607690 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65ffcb6-1fd6-4feb-acb5-ea398ff1d2fc None admin] RBAC: Adding query filter params (name=admin) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:04:52.607995 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65ffcb6-1fd6-4feb-acb5-ea398ff1d2fc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:52.609475 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65ffcb6-1fd6-4feb-acb5-ea398ff1d2fc None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:52.618873 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 404/836] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Aug 31 01:04:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:04:52.625849 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b6ff4e-97c0-4049-9135-636b12a5ee24 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:52.693356 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:52.732759 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b6ff4e-97c0-4049-9135-636b12a5ee24 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:52.760443 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc183b9-8731-42f1-9f70-07ca533bda01 None None] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:52.760788 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc183b9-8731-42f1-9f70-07ca533bda01 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:52.760972 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc183b9-8731-42f1-9f70-07ca533bda01 None None] PATH_INFO: `/` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:52.762558 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 405/837] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:04:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:04:52.767366 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe26d016-bda5-40d2-8fcf-35208f0b2164 None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:52.767636 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe26d016-bda5-40d2-8fcf-35208f0b2164 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:52.767935 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe26d016-bda5-40d2-8fcf-35208f0b2164 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:52.770575 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c2dd39-dcba-4376-8231-9fd7b3d6bce2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:52.771314 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c2dd39-dcba-4376-8231-9fd7b3d6bce2 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:52.771697 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c2dd39-dcba-4376-8231-9fd7b3d6bce2 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:52.772085 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c2dd39-dcba-4376-8231-9fd7b3d6bce2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:52.784295 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c2dd39-dcba-4376-8231-9fd7b3d6bce2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:52.785905 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c2dd39-dcba-4376-8231-9fd7b3d6bce2 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:52.786412 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-fe26d016-bda5-40d2-8fcf-35208f0b2164 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:04:52.787611 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-fe26d016-bda5-40d2-8fcf-35208f0b2164 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:04:52.801518 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application [None req-c4c2dd39-dcba-4376-8231-9fd7b3d6bce2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 01:04:52.801518 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:04:52.801518 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:04:52.801518 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:04:52.801518 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:04:52.801518 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:04:52.801518 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:04:52.801518 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:04:52.801518 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:04:52.801518 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:04:52.801518 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:04:52.801518 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:04:52.801518 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 01:04:52.801518 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 01:04:52.801518 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 01:04:52.801518 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 01:04:52.802458 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:04:52.802458 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:04:52.802458 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:04:52.802458 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:04:52.802458 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:04:52.802458 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:04:52.802458 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:04:52.802458 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application with Lock( Aug 31 01:04:52.802458 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:04:52.802458 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:04:52.802458 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:04:52.802458 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:04:52.802458 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:04:52.802458 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:04:52.802458 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:04:52.802458 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:04:52.802458 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 01:04:52.803345 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 01:04:52.803345 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 01:04:52.803345 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Aug 31 01:04:52.805941 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 433/838] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 01:04:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:04:52.811835 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c2dd39-dcba-4376-8231-9fd7b3d6bce2 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:52.869022 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 406/839] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:04:52 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 01:04:52.916140 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c2dd39-dcba-4376-8231-9fd7b3d6bce2 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:52.949990 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:52.953356 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346d369b-9789-4c13-b475-a5a6fea3a644 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:52.953984 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346d369b-9789-4c13-b475-a5a6fea3a644 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:52.954341 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346d369b-9789-4c13-b475-a5a6fea3a644 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:52.954619 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346d369b-9789-4c13-b475-a5a6fea3a644 None admin] PATH_INFO: `/v3/domains` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:52.955264 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346d369b-9789-4c13-b475-a5a6fea3a644 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:04:52.955564 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346d369b-9789-4c13-b475-a5a6fea3a644 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:52.956963 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346d369b-9789-4c13-b475-a5a6fea3a644 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:52.969304 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 434/840] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 31 01:04:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:04:52.974970 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe26d016-bda5-40d2-8fcf-35208f0b2164 None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:53.080903 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe26d016-bda5-40d2-8fcf-35208f0b2164 None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:53.102085 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:53.115616 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2a4394-9bf8-4dce-bc7c-d31c58bd0ca5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:53.116811 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2a4394-9bf8-4dce-bc7c-d31c58bd0ca5 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:53.117094 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2a4394-9bf8-4dce-bc7c-d31c58bd0ca5 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:53.117369 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2a4394-9bf8-4dce-bc7c-d31c58bd0ca5 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:53.124352 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2a4394-9bf8-4dce-bc7c-d31c58bd0ca5 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:53.125811 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2a4394-9bf8-4dce-bc7c-d31c58bd0ca5 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:53.137463 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application [None req-bb2a4394-9bf8-4dce-bc7c-d31c58bd0ca5 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 31 01:04:53.137463 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:04:53.137463 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:04:53.137463 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:04:53.137463 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:04:53.137463 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:04:53.137463 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:04:53.137463 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:04:53.137463 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:04:53.137463 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:04:53.137463 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:04:53.137463 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:04:53.137463 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 01:04:53.137463 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 01:04:53.137463 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 01:04:53.137463 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 01:04:53.138546 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:04:53.138546 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:04:53.138546 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 01:04:53.138546 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 01:04:53.138546 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 01:04:53.138546 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 01:04:53.138546 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:04:53.138546 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:04:53.138546 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:04:53.138546 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:04:53.138546 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:04:53.138546 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application with Lock( Aug 31 01:04:53.138546 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:04:53.138546 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:04:53.138546 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:04:53.138546 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:04:53.138546 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:04:53.138546 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:04:53.139490 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:04:53.139490 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:04:53.139490 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 01:04:53.139490 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 01:04:53.139490 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 01:04:53.139490 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 01:04:53.139490 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 01:04:53.139490 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 01:04:53.139490 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 31 01:04:53.139490 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Aug 31 01:04:53.146212 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 407/841] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Aug 31 01:04:52 2022] GET /identity/v3/users/swift => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:04:53.151648 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346d369b-9789-4c13-b475-a5a6fea3a644 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:53.159475 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2a4394-9bf8-4dce-bc7c-d31c58bd0ca5 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:53.253876 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346d369b-9789-4c13-b475-a5a6fea3a644 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:53.267069 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2a4394-9bf8-4dce-bc7c-d31c58bd0ca5 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:53.291079 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d96586a-209b-4334-ba2b-2d018fa2c8a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:53.291686 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d96586a-209b-4334-ba2b-2d018fa2c8a7 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:53.291960 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d96586a-209b-4334-ba2b-2d018fa2c8a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:53.292234 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d96586a-209b-4334-ba2b-2d018fa2c8a7 None admin] PATH_INFO: `/v3/users` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:53.293002 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d96586a-209b-4334-ba2b-2d018fa2c8a7 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:04:53.293304 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d96586a-209b-4334-ba2b-2d018fa2c8a7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:53.294786 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d96586a-209b-4334-ba2b-2d018fa2c8a7 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:53.309020 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a3228c-7417-4f74-8287-a4c09d81477a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:53.309667 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a3228c-7417-4f74-8287-a4c09d81477a None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:53.310222 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a3228c-7417-4f74-8287-a4c09d81477a None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:53.310510 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a3228c-7417-4f74-8287-a4c09d81477a None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:53.311114 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a3228c-7417-4f74-8287-a4c09d81477a None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:53.312636 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a3228c-7417-4f74-8287-a4c09d81477a None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:53.317013 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 435/842] 10.222.0.20 () {56 vars in 1135 bytes} [Wed Aug 31 01:04:53 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:04:53.318073 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d96586a-209b-4334-ba2b-2d018fa2c8a7 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:53.322979 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application [None req-b5a3228c-7417-4f74-8287-a4c09d81477a None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 31 01:04:53.322979 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:04:53.322979 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 01:04:53.322979 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 01:04:53.322979 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 01:04:53.322979 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 01:04:53.322979 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 01:04:53.322979 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 01:04:53.322979 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 31 01:04:53.322979 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Aug 31 01:04:53.322979 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 01:04:53.322979 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Aug 31 01:04:53.322979 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:04:53.322979 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:04:53.322979 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:04:53.322979 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:04:53.322979 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:04:53.323976 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:04:53.323976 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:04:53.323976 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:04:53.323976 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:04:53.323976 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:04:53.323976 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:04:53.323976 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 01:04:53.323976 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 01:04:53.323976 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 01:04:53.323976 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 01:04:53.323976 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:04:53.323976 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:04:53.323976 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:04:53.323976 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:04:53.323976 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:04:53.323976 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:04:53.325133 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:04:53.325133 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application with Lock( Aug 31 01:04:53.325133 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:04:53.325133 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:04:53.325133 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:04:53.325133 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:04:53.325133 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:04:53.325133 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:04:53.325133 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:04:53.325133 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:04:53.325133 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 01:04:53.325133 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 01:04:53.325133 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 31 01:04:53.325133 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Aug 31 01:04:53.328814 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 408/843] 10.222.0.20 () {56 vars in 1114 bytes} [Wed Aug 31 01:04:53 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:04:53.330646 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a3228c-7417-4f74-8287-a4c09d81477a None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:53.362155 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:53.387424 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:53.436775 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d96586a-209b-4334-ba2b-2d018fa2c8a7 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:53.447449 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a3228c-7417-4f74-8287-a4c09d81477a None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:53.474860 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532f840c-bfca-4902-be63-521ebae8968c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:53.476029 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532f840c-bfca-4902-be63-521ebae8968c None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:53.476317 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532f840c-bfca-4902-be63-521ebae8968c None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:53.476588 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532f840c-bfca-4902-be63-521ebae8968c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:53.486205 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1770347-0aaf-49c4-89c6-2d5e23ef5547 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:53.486836 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1770347-0aaf-49c4-89c6-2d5e23ef5547 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:53.487120 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1770347-0aaf-49c4-89c6-2d5e23ef5547 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:53.487394 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1770347-0aaf-49c4-89c6-2d5e23ef5547 None admin] PATH_INFO: `/v3/services` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:53.488103 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1770347-0aaf-49c4-89c6-2d5e23ef5547 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:04:53.488323 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1770347-0aaf-49c4-89c6-2d5e23ef5547 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:53.489722 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1770347-0aaf-49c4-89c6-2d5e23ef5547 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:53.494222 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 409/844] 10.222.0.20 () {56 vars in 1114 bytes} [Wed Aug 31 01:04:53 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:04:53.497923 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d012fc18-701b-45ca-8fc3-f9379368a718 None None] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:53.498207 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d012fc18-701b-45ca-8fc3-f9379368a718 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:53.498479 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d012fc18-701b-45ca-8fc3-f9379368a718 None None] PATH_INFO: `/` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:53.502459 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 410/845] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:04:53 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:04:53.503184 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d012fc18-701b-45ca-8fc3-f9379368a718 None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:53.554450 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532f840c-bfca-4902-be63-521ebae8968c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:53.555812 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532f840c-bfca-4902-be63-521ebae8968c None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:53.590521 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:53.613256 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d012fc18-701b-45ca-8fc3-f9379368a718 None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:53.622414 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:53.648367 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec26e7b-44cc-4129-a918-05e708f32dd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:53.649001 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec26e7b-44cc-4129-a918-05e708f32dd7 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:53.649285 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec26e7b-44cc-4129-a918-05e708f32dd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:53.649556 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec26e7b-44cc-4129-a918-05e708f32dd7 None admin] PATH_INFO: `/v3/services` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:53.650212 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec26e7b-44cc-4129-a918-05e708f32dd7 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:04:53.650553 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec26e7b-44cc-4129-a918-05e708f32dd7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:53.652837 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec26e7b-44cc-4129-a918-05e708f32dd7 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:53.664754 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 411/846] 10.222.0.20 () {56 vars in 1114 bytes} [Wed Aug 31 01:04:53 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:04:53.664754 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc82720-2d6b-4e38-9c3f-137255c58cf6 None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:53.664754 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc82720-2d6b-4e38-9c3f-137255c58cf6 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:53.664754 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc82720-2d6b-4e38-9c3f-137255c58cf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:53.677439 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 436/847] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Aug 31 01:04:53 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 31 01:04:53.678627 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532f840c-bfca-4902-be63-521ebae8968c None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:53.678812 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-bcc82720-2d6b-4e38-9c3f-137255c58cf6 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:04:53.680038 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-bcc82720-2d6b-4e38-9c3f-137255c58cf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:04:53.761180 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 412/848] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:04:53 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 01:04:53.806444 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532f840c-bfca-4902-be63-521ebae8968c None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:53.845010 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f522cb0-b2b0-4cd1-a4a7-0328d32c4287 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:53.845651 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f522cb0-b2b0-4cd1-a4a7-0328d32c4287 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:53.845933 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f522cb0-b2b0-4cd1-a4a7-0328d32c4287 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:53.846207 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f522cb0-b2b0-4cd1-a4a7-0328d32c4287 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:53.848348 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:53.857244 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f522cb0-b2b0-4cd1-a4a7-0328d32c4287 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:53.858689 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f522cb0-b2b0-4cd1-a4a7-0328d32c4287 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:53.873332 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application [None req-8f522cb0-b2b0-4cd1-a4a7-0328d32c4287 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 01:04:53.873332 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:04:53.873332 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:04:53.873332 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:04:53.873332 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:04:53.873332 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:04:53.873332 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:04:53.873332 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:04:53.873332 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:04:53.873332 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:04:53.873332 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:04:53.873332 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:04:53.873332 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 01:04:53.873332 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 01:04:53.873332 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 01:04:53.873332 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 01:04:53.874325 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:04:53.874325 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:04:53.874325 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:04:53.874325 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:04:53.874325 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:04:53.874325 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:04:53.874325 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:04:53.874325 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application with Lock( Aug 31 01:04:53.874325 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:04:53.874325 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:04:53.874325 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:04:53.874325 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:04:53.874325 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:04:53.874325 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:04:53.874325 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:04:53.874325 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:04:53.874325 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 01:04:53.875205 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 01:04:53.875205 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 01:04:53.875205 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Aug 31 01:04:53.875205 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 437/849] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 01:04:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:04:53.880570 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc82720-2d6b-4e38-9c3f-137255c58cf6 None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:53.998254 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc82720-2d6b-4e38-9c3f-137255c58cf6 None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:53.999819 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:54.029088 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f522cb0-b2b0-4cd1-a4a7-0328d32c4287 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:54.034197 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ff72b5-743d-4d71-bf37-5b6ee696c6bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:54.034810 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ff72b5-743d-4d71-bf37-5b6ee696c6bf None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:54.035133 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ff72b5-743d-4d71-bf37-5b6ee696c6bf None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:54.035412 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ff72b5-743d-4d71-bf37-5b6ee696c6bf None admin] PATH_INFO: `/v3/domains` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:54.036054 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ff72b5-743d-4d71-bf37-5b6ee696c6bf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:04:54.036356 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ff72b5-743d-4d71-bf37-5b6ee696c6bf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:54.037777 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ff72b5-743d-4d71-bf37-5b6ee696c6bf None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:54.049984 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 413/850] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 31 01:04:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:04:54.056627 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ff72b5-743d-4d71-bf37-5b6ee696c6bf None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:54.158754 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f522cb0-b2b0-4cd1-a4a7-0328d32c4287 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:54.177024 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ff72b5-743d-4d71-bf37-5b6ee696c6bf None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:54.204697 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253402b8-ca76-4128-a551-c7f9cb78cc76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:54.205961 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253402b8-ca76-4128-a551-c7f9cb78cc76 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:54.206859 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253402b8-ca76-4128-a551-c7f9cb78cc76 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:54.207577 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253402b8-ca76-4128-a551-c7f9cb78cc76 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:54.208597 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253402b8-ca76-4128-a551-c7f9cb78cc76 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:54.210996 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253402b8-ca76-4128-a551-c7f9cb78cc76 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:54.214366 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e71b5be-d271-436b-8a03-b54728dcfbfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:54.215210 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e71b5be-d271-436b-8a03-b54728dcfbfb None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:54.215569 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e71b5be-d271-436b-8a03-b54728dcfbfb None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:54.216009 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e71b5be-d271-436b-8a03-b54728dcfbfb None admin] PATH_INFO: `/v3/projects/service` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:54.222848 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application [None req-253402b8-ca76-4128-a551-c7f9cb78cc76 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 31 01:04:54.222848 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:04:54.222848 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 01:04:54.222848 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 01:04:54.222848 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 01:04:54.222848 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 01:04:54.222848 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 01:04:54.222848 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 01:04:54.222848 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 31 01:04:54.222848 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Aug 31 01:04:54.222848 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 01:04:54.222848 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Aug 31 01:04:54.222848 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:04:54.222848 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:04:54.222848 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:04:54.222848 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:04:54.223794 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:04:54.223794 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:04:54.223794 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:04:54.223794 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:04:54.223794 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:04:54.223794 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:04:54.223794 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:04:54.223794 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 01:04:54.223794 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 01:04:54.223794 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 01:04:54.223794 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 01:04:54.223794 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:04:54.223794 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:04:54.223794 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:04:54.223794 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:04:54.223794 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:04:54.223794 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:04:54.224670 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:04:54.224670 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application with Lock( Aug 31 01:04:54.224670 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:04:54.224670 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:04:54.224670 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:04:54.224670 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:04:54.224670 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:04:54.224670 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:04:54.224670 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:04:54.224670 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:04:54.224670 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 01:04:54.224670 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 01:04:54.224670 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 31 01:04:54.224670 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Aug 31 01:04:54.225574 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e71b5be-d271-436b-8a03-b54728dcfbfb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:54.225574 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e71b5be-d271-436b-8a03-b54728dcfbfb None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:54.230875 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 438/851] 10.222.0.20 () {56 vars in 1132 bytes} [Wed Aug 31 01:04:54 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 01:04:54.232220 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253402b8-ca76-4128-a551-c7f9cb78cc76 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:54.239952 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application [None req-9e71b5be-d271-436b-8a03-b54728dcfbfb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 01:04:54.239952 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:04:54.239952 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:04:54.239952 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:04:54.239952 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:04:54.239952 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:04:54.239952 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:04:54.239952 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:04:54.239952 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:04:54.239952 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:04:54.239952 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:04:54.239952 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:04:54.239952 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 01:04:54.239952 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 01:04:54.239952 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 01:04:54.241414 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 01:04:54.241414 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:04:54.241414 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:04:54.241414 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:04:54.241414 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:04:54.241414 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:04:54.241414 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:04:54.241414 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:04:54.241414 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application with Lock( Aug 31 01:04:54.241414 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:04:54.241414 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:04:54.241414 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:04:54.241414 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:04:54.241414 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:04:54.241414 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:04:54.241414 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:04:54.241414 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:04:54.242324 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 01:04:54.242324 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 01:04:54.242324 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 01:04:54.242324 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 01:04:54.242324 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 01:04:54.242324 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 01:04:54.242324 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 01:04:54.242324 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Aug 31 01:04:54.249652 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 414/852] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Aug 31 01:04:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 01:04:54.251588 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e71b5be-d271-436b-8a03-b54728dcfbfb None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:54.283422 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:54.293622 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:54.343598 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253402b8-ca76-4128-a551-c7f9cb78cc76 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:54.377829 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dedabe-367f-4a4f-86c3-1d471f0f4995 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:54.378445 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dedabe-367f-4a4f-86c3-1d471f0f4995 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:54.378728 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dedabe-367f-4a4f-86c3-1d471f0f4995 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:54.379002 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dedabe-367f-4a4f-86c3-1d471f0f4995 None admin] PATH_INFO: `/v3/services` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:54.379629 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0dedabe-367f-4a4f-86c3-1d471f0f4995 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:04:54.379932 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0dedabe-367f-4a4f-86c3-1d471f0f4995 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:54.381341 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0dedabe-367f-4a4f-86c3-1d471f0f4995 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:54.382635 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e71b5be-d271-436b-8a03-b54728dcfbfb None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:54.385979 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 439/853] 10.222.0.20 () {56 vars in 1126 bytes} [Wed Aug 31 01:04:54 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:04:54.387145 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb6a68e-118c-4731-9441-c96316f8a3d8 None None] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:54.387430 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb6a68e-118c-4731-9441-c96316f8a3d8 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:54.387705 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb6a68e-118c-4731-9441-c96316f8a3d8 None None] PATH_INFO: `/` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:54.392843 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 440/854] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:04:54 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:04:54.393653 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb6a68e-118c-4731-9441-c96316f8a3d8 None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:54.425327 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185d4db7-0b73-4585-897e-77362542ee49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:54.426138 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185d4db7-0b73-4585-897e-77362542ee49 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:54.426581 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185d4db7-0b73-4585-897e-77362542ee49 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:54.427008 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185d4db7-0b73-4585-897e-77362542ee49 None admin] PATH_INFO: `/v3/projects` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:54.427838 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185d4db7-0b73-4585-897e-77362542ee49 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:04:54.428282 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185d4db7-0b73-4585-897e-77362542ee49 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:54.430431 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185d4db7-0b73-4585-897e-77362542ee49 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:54.442224 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 415/855] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Aug 31 01:04:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:04:54.443636 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1133d6e-873e-4001-b093-fad1918ebe75 None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:54.443918 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1133d6e-873e-4001-b093-fad1918ebe75 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:54.444216 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1133d6e-873e-4001-b093-fad1918ebe75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:54.463082 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-d1133d6e-873e-4001-b093-fad1918ebe75 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:04:54.464472 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-d1133d6e-873e-4001-b093-fad1918ebe75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:04:54.503475 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb6a68e-118c-4731-9441-c96316f8a3d8 None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:54.517234 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:54.544752 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 416/856] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:04:54 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 01:04:54.544752 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1133d6e-873e-4001-b093-fad1918ebe75 None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:54.547616 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61245fa5-8681-4dde-b915-cc005c815f80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:54.548283 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61245fa5-8681-4dde-b915-cc005c815f80 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:54.548627 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61245fa5-8681-4dde-b915-cc005c815f80 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:54.548980 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61245fa5-8681-4dde-b915-cc005c815f80 None admin] PATH_INFO: `/v3/services` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:54.549605 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61245fa5-8681-4dde-b915-cc005c815f80 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:04:54.549908 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61245fa5-8681-4dde-b915-cc005c815f80 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:54.551350 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61245fa5-8681-4dde-b915-cc005c815f80 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:54.556920 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 441/857] 10.222.0.20 () {56 vars in 1126 bytes} [Wed Aug 31 01:04:54 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:04:54.565123 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:54.662129 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1133d6e-873e-4001-b093-fad1918ebe75 None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:54.701396 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe63dbff-341e-453e-a767-909cd4b206b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:54.702008 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe63dbff-341e-453e-a767-909cd4b206b2 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:54.702291 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe63dbff-341e-453e-a767-909cd4b206b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:54.702569 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe63dbff-341e-453e-a767-909cd4b206b2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:54.703302 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe63dbff-341e-453e-a767-909cd4b206b2 None admin] RBAC: Adding query filter params (role.id=d009411496b8452b88e77ffb8db015a7, scope.project.id=60b2abc15c2849ba8a1ed92117cc981b, user.id=fc1d010919ec40878ae829cc8dceec4c) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:04:54.703617 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe63dbff-341e-453e-a767-909cd4b206b2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:54.705078 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe63dbff-341e-453e-a767-909cd4b206b2 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:54.712019 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 417/858] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Aug 31 01:04:54 2022] GET /identity/v3/role_assignments?user.id=fc1d010919ec40878ae829cc8dceec4c&scope.project.id=60b2abc15c2849ba8a1ed92117cc981b&role.id=d009411496b8452b88e77ffb8db015a7 => generated 608 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:04:54.833187 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61245fa5-8681-4dde-b915-cc005c815f80 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:54.956910 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61245fa5-8681-4dde-b915-cc005c815f80 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:54.992102 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c488d5b-cc24-49d3-8d65-31f191570127 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:54.992750 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c488d5b-cc24-49d3-8d65-31f191570127 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:54.993041 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c488d5b-cc24-49d3-8d65-31f191570127 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:54.993315 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c488d5b-cc24-49d3-8d65-31f191570127 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:55.003749 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c488d5b-cc24-49d3-8d65-31f191570127 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 01:04:55.009177 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application [None req-2c488d5b-cc24-49d3-8d65-31f191570127 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 01:04:55.009177 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:04:55.009177 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:04:55.009177 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:04:55.009177 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:04:55.009177 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:04:55.009177 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:04:55.009177 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:04:55.009177 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:04:55.009177 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:04:55.009177 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:04:55.009177 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:04:55.009177 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 01:04:55.009177 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 01:04:55.009177 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 01:04:55.009177 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 01:04:55.010114 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 01:04:55.010114 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 01:04:55.010114 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 01:04:55.010114 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 01:04:55.010114 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:04:55.010114 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:04:55.010114 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:04:55.010114 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:04:55.010114 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:04:55.010114 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:04:55.010114 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:04:55.010114 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application with Lock( Aug 31 01:04:55.010114 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:04:55.010114 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:04:55.010114 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:04:55.010114 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:04:55.011149 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:04:55.011149 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:04:55.011149 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:04:55.011149 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:04:55.011149 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 01:04:55.011149 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 01:04:55.011149 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 01:04:55.011149 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 01:04:55.011149 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 01:04:55.011149 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 01:04:55.011149 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 01:04:55.011149 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Aug 31 01:04:55.011149 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 442/859] 10.222.0.20 () {56 vars in 1102 bytes} [Wed Aug 31 01:04:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:04:55.016086 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe63dbff-341e-453e-a767-909cd4b206b2 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:55.098501 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:55.121754 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe63dbff-341e-453e-a767-909cd4b206b2 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:55.156994 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8bd607-ace2-4c38-89e4-cdc0c4784029 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:55.157603 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8bd607-ace2-4c38-89e4-cdc0c4784029 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:55.157884 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8bd607-ace2-4c38-89e4-cdc0c4784029 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:55.158161 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8bd607-ace2-4c38-89e4-cdc0c4784029 None admin] PATH_INFO: `/v3/roles` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:55.158833 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8bd607-ace2-4c38-89e4-cdc0c4784029 None admin] RBAC: Adding query filter params (name=service) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:04:55.159141 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8bd607-ace2-4c38-89e4-cdc0c4784029 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:55.160524 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8bd607-ace2-4c38-89e4-cdc0c4784029 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:55.169908 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 418/860] 10.222.0.20 () {56 vars in 1103 bytes} [Wed Aug 31 01:04:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:04:55.175154 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c488d5b-cc24-49d3-8d65-31f191570127 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:55.224605 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:55.281654 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c488d5b-cc24-49d3-8d65-31f191570127 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:55.315882 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6732b6-00b4-45c3-be7c-0e417aedc43b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:55.316512 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6732b6-00b4-45c3-be7c-0e417aedc43b None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:55.316814 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6732b6-00b4-45c3-be7c-0e417aedc43b None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:55.317262 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6732b6-00b4-45c3-be7c-0e417aedc43b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:55.328500 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6732b6-00b4-45c3-be7c-0e417aedc43b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:55.329964 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6732b6-00b4-45c3-be7c-0e417aedc43b None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:55.344248 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application [None req-3b6732b6-00b4-45c3-be7c-0e417aedc43b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 01:04:55.344248 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:04:55.344248 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:04:55.344248 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:04:55.344248 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:04:55.344248 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:04:55.344248 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:04:55.344248 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:04:55.344248 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:04:55.344248 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:04:55.344248 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:04:55.344248 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:04:55.344248 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 01:04:55.344248 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 01:04:55.344248 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 01:04:55.344248 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 01:04:55.345225 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:04:55.345225 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:04:55.345225 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:04:55.345225 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:04:55.345225 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:04:55.345225 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:04:55.345225 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:04:55.345225 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application with Lock( Aug 31 01:04:55.345225 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:04:55.345225 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:04:55.345225 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:04:55.345225 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:04:55.345225 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:04:55.345225 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:04:55.345225 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:04:55.345225 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:04:55.345225 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 01:04:55.346115 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 01:04:55.346115 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 01:04:55.346115 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Aug 31 01:04:55.346115 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 443/861] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 01:04:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:04:55.352787 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8bd607-ace2-4c38-89e4-cdc0c4784029 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:55.454535 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8bd607-ace2-4c38-89e4-cdc0c4784029 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:55.489850 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc47944-0103-41c2-a5f2-b81c09f5ca2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:55.490383 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc47944-0103-41c2-a5f2-b81c09f5ca2f None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:55.490668 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc47944-0103-41c2-a5f2-b81c09f5ca2f None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:55.491468 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc47944-0103-41c2-a5f2-b81c09f5ca2f None admin] PATH_INFO: `/v3/domains` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:55.492116 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc47944-0103-41c2-a5f2-b81c09f5ca2f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:04:55.492423 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc47944-0103-41c2-a5f2-b81c09f5ca2f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:55.493827 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc47944-0103-41c2-a5f2-b81c09f5ca2f None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:55.500969 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:55.506225 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 419/862] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 31 01:04:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:04:55.512286 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6732b6-00b4-45c3-be7c-0e417aedc43b None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:55.605566 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:55.615014 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6732b6-00b4-45c3-be7c-0e417aedc43b None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:55.650216 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81da6aa2-2f4f-425b-b5ad-8d557602ffa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:55.650848 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81da6aa2-2f4f-425b-b5ad-8d557602ffa6 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:55.651130 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81da6aa2-2f4f-425b-b5ad-8d557602ffa6 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:55.651402 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81da6aa2-2f4f-425b-b5ad-8d557602ffa6 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:55.658953 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81da6aa2-2f4f-425b-b5ad-8d557602ffa6 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:55.660390 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81da6aa2-2f4f-425b-b5ad-8d557602ffa6 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:55.671646 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application [None req-81da6aa2-2f4f-425b-b5ad-8d557602ffa6 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 31 01:04:55.671646 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:04:55.671646 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:04:55.671646 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:04:55.671646 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:04:55.671646 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:04:55.671646 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:04:55.671646 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:04:55.671646 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:04:55.671646 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:04:55.671646 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:04:55.671646 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:04:55.671646 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 01:04:55.671646 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 01:04:55.671646 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 01:04:55.671646 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 01:04:55.672592 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:04:55.672592 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:04:55.672592 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 01:04:55.672592 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 01:04:55.672592 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 01:04:55.672592 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 01:04:55.672592 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:04:55.672592 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:04:55.672592 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:04:55.672592 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:04:55.672592 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:04:55.672592 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application with Lock( Aug 31 01:04:55.672592 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:04:55.672592 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:04:55.672592 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:04:55.672592 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:04:55.672592 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:04:55.672592 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:04:55.673565 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:04:55.673565 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:04:55.673565 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 01:04:55.673565 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 01:04:55.673565 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 01:04:55.673565 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 01:04:55.673565 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 01:04:55.673565 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 01:04:55.673565 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 31 01:04:55.673565 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Aug 31 01:04:55.673565 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 444/863] 10.222.0.20 () {56 vars in 1117 bytes} [Wed Aug 31 01:04:55 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:04:55.680945 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc47944-0103-41c2-a5f2-b81c09f5ca2f None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:55.742336 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944f0527-0f9f-477f-a99c-a307bbb5ca82 None None] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:55.742648 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944f0527-0f9f-477f-a99c-a307bbb5ca82 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:55.742833 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944f0527-0f9f-477f-a99c-a307bbb5ca82 None None] PATH_INFO: `/` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:55.744247 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 445/864] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:04:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:04:55.749055 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cd1092-7508-4c03-b23a-a6b034ad0020 None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:55.749338 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cd1092-7508-4c03-b23a-a6b034ad0020 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:55.749613 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cd1092-7508-4c03-b23a-a6b034ad0020 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:55.768301 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-51cd1092-7508-4c03-b23a-a6b034ad0020 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:04:55.769510 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-51cd1092-7508-4c03-b23a-a6b034ad0020 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:04:55.793661 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc47944-0103-41c2-a5f2-b81c09f5ca2f None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:55.834143 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463eae3a-65f3-4b71-af09-0e84a2a1460e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:55.834913 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463eae3a-65f3-4b71-af09-0e84a2a1460e None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:55.835301 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463eae3a-65f3-4b71-af09-0e84a2a1460e None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:55.835579 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463eae3a-65f3-4b71-af09-0e84a2a1460e None admin] PATH_INFO: `/v3/users` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:55.836300 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463eae3a-65f3-4b71-af09-0e84a2a1460e None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:04:55.836602 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463eae3a-65f3-4b71-af09-0e84a2a1460e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:55.838736 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463eae3a-65f3-4b71-af09-0e84a2a1460e None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:55.846364 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 446/865] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:04:55 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 01:04:55.861197 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 420/866] 10.222.0.20 () {56 vars in 1149 bytes} [Wed Aug 31 01:04:55 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:04:55.875110 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cd1092-7508-4c03-b23a-a6b034ad0020 None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:55.958795 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:55.978477 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cd1092-7508-4c03-b23a-a6b034ad0020 None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:56.013737 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e2a9f3-245a-488c-83c0-97b15fdc0e77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:56.014273 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e2a9f3-245a-488c-83c0-97b15fdc0e77 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:56.014551 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e2a9f3-245a-488c-83c0-97b15fdc0e77 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:56.014824 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e2a9f3-245a-488c-83c0-97b15fdc0e77 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:56.087391 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e2a9f3-245a-488c-83c0-97b15fdc0e77 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:56.088964 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e2a9f3-245a-488c-83c0-97b15fdc0e77 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:56.159472 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463eae3a-65f3-4b71-af09-0e84a2a1460e None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:56.169051 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:56.190495 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 447/867] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Aug 31 01:04:55 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 01:04:56.196396 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e2a9f3-245a-488c-83c0-97b15fdc0e77 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:56.227115 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:56.275342 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463eae3a-65f3-4b71-af09-0e84a2a1460e None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:56.305328 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e2a9f3-245a-488c-83c0-97b15fdc0e77 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:56.316241 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173d353e-59e1-47ac-9b58-d24f4f71712d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:56.316241 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173d353e-59e1-47ac-9b58-d24f4f71712d None admin] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:56.316241 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173d353e-59e1-47ac-9b58-d24f4f71712d None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:56.316241 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173d353e-59e1-47ac-9b58-d24f4f71712d None admin] PATH_INFO: `/v3/services` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:56.316628 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173d353e-59e1-47ac-9b58-d24f4f71712d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:56.318028 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173d353e-59e1-47ac-9b58-d24f4f71712d None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:56.343925 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23576361-7fb7-4f87-a770-2c9d18905ff4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:56.344558 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23576361-7fb7-4f87-a770-2c9d18905ff4 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:56.344922 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23576361-7fb7-4f87-a770-2c9d18905ff4 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:56.345199 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23576361-7fb7-4f87-a770-2c9d18905ff4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:56.356476 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23576361-7fb7-4f87-a770-2c9d18905ff4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:56.357996 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23576361-7fb7-4f87-a770-2c9d18905ff4 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:56.372228 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application [None req-23576361-7fb7-4f87-a770-2c9d18905ff4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 01:04:56.372228 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:04:56.372228 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:04:56.372228 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:04:56.372228 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:04:56.372228 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:04:56.372228 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:04:56.372228 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:04:56.372228 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:04:56.372228 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:04:56.372228 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:04:56.372228 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:04:56.372228 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 01:04:56.372228 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 01:04:56.372228 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 01:04:56.372228 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 01:04:56.379051 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:04:56.379051 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:04:56.379051 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:04:56.379051 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:04:56.379051 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:04:56.379051 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:04:56.379051 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:04:56.379051 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application with Lock( Aug 31 01:04:56.379051 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:04:56.379051 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:04:56.379051 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:04:56.379051 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:04:56.379051 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:04:56.379051 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:04:56.379051 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:04:56.379051 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:04:56.379051 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 01:04:56.379984 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 01:04:56.379984 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 01:04:56.379984 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Aug 31 01:04:56.379984 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 448/868] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 01:04:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:04:56.379984 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23576361-7fb7-4f87-a770-2c9d18905ff4 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:56.411234 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:56.442449 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 421/869] 10.222.0.20 () {60 vars in 1141 bytes} [Wed Aug 31 01:04:56 2022] POST /identity/v3/services => generated 251 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:04:56.497735 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23576361-7fb7-4f87-a770-2c9d18905ff4 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:56.542974 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550faebc-c9cc-4d4e-9b0f-bd600d93b086 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:56.543577 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550faebc-c9cc-4d4e-9b0f-bd600d93b086 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:56.543863 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550faebc-c9cc-4d4e-9b0f-bd600d93b086 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:56.544134 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550faebc-c9cc-4d4e-9b0f-bd600d93b086 None admin] PATH_INFO: `/v3/domains` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:56.544810 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550faebc-c9cc-4d4e-9b0f-bd600d93b086 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:04:56.545116 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550faebc-c9cc-4d4e-9b0f-bd600d93b086 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:56.546508 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550faebc-c9cc-4d4e-9b0f-bd600d93b086 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:56.560561 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 449/870] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 31 01:04:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 01:04:56.569932 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173d353e-59e1-47ac-9b58-d24f4f71712d None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:56.613816 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c7a345-02e0-4da4-be2e-09affcc13cfa None None] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:56.614124 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c7a345-02e0-4da4-be2e-09affcc13cfa None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:56.614379 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c7a345-02e0-4da4-be2e-09affcc13cfa None None] PATH_INFO: `/` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:56.615880 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 450/871] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:04:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:04:56.620805 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a115a3d4-0606-455d-8d43-af4a6d828e86 None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:56.621093 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a115a3d4-0606-455d-8d43-af4a6d828e86 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:56.621370 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a115a3d4-0606-455d-8d43-af4a6d828e86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:56.646845 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-a115a3d4-0606-455d-8d43-af4a6d828e86 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:04:56.646845 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-a115a3d4-0606-455d-8d43-af4a6d828e86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:04:56.682971 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173d353e-59e1-47ac-9b58-d24f4f71712d None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:56.700551 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:56.716103 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 451/872] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:04:56 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 01:04:56.722410 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b6e939-8748-46da-8540-8eb82d732f7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:56.723059 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b6e939-8748-46da-8540-8eb82d732f7e None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:56.723337 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b6e939-8748-46da-8540-8eb82d732f7e None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:56.723612 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b6e939-8748-46da-8540-8eb82d732f7e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:56.732027 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b6e939-8748-46da-8540-8eb82d732f7e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:56.733511 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b6e939-8748-46da-8540-8eb82d732f7e None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:56.745605 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application [None req-78b6e939-8748-46da-8540-8eb82d732f7e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 01:04:56.745605 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:04:56.745605 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:04:56.745605 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:04:56.745605 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:04:56.745605 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:04:56.745605 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:04:56.745605 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:04:56.745605 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:04:56.745605 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:04:56.745605 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:04:56.745605 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:04:56.745605 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 01:04:56.745605 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 01:04:56.745605 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 01:04:56.746539 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 01:04:56.746539 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:04:56.746539 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:04:56.746539 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:04:56.746539 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:04:56.746539 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:04:56.746539 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:04:56.746539 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:04:56.746539 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application with Lock( Aug 31 01:04:56.746539 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:04:56.746539 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:04:56.746539 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:04:56.746539 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:04:56.746539 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:04:56.746539 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:04:56.746539 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:04:56.746539 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:04:56.747409 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 01:04:56.747409 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 01:04:56.747409 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 01:04:56.747409 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 01:04:56.747409 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 01:04:56.747409 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 01:04:56.747409 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 01:04:56.747409 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Aug 31 01:04:56.747409 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 422/873] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Aug 31 01:04:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:04:56.754118 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a115a3d4-0606-455d-8d43-af4a6d828e86 None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:56.781132 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:56.825792 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df82923-e943-46ba-88e3-bc014d5f7547 None None] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:56.826082 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df82923-e943-46ba-88e3-bc014d5f7547 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:56.826274 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df82923-e943-46ba-88e3-bc014d5f7547 None None] PATH_INFO: `/` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:56.827768 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 423/874] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:04:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:04:56.834356 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebd880a-b719-4b9f-b87a-2a4a8d4f41b5 None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:56.834957 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebd880a-b719-4b9f-b87a-2a4a8d4f41b5 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:56.835444 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebd880a-b719-4b9f-b87a-2a4a8d4f41b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:56.859482 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-4ebd880a-b719-4b9f-b87a-2a4a8d4f41b5 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:04:56.861811 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-4ebd880a-b719-4b9f-b87a-2a4a8d4f41b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:04:56.868593 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a115a3d4-0606-455d-8d43-af4a6d828e86 None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:56.919967 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3d33e5-ede4-492f-b7a8-8d82cf54c431 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:56.921499 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3d33e5-ede4-492f-b7a8-8d82cf54c431 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:56.921988 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3d33e5-ede4-492f-b7a8-8d82cf54c431 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:56.922458 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3d33e5-ede4-492f-b7a8-8d82cf54c431 None admin] PATH_INFO: `/v3/projects` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:56.923824 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3d33e5-ede4-492f-b7a8-8d82cf54c431 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:04:56.924354 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3d33e5-ede4-492f-b7a8-8d82cf54c431 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:56.926691 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3d33e5-ede4-492f-b7a8-8d82cf54c431 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:56.954013 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 424/875] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:04:56 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 01:04:56.954214 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 452/876] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Aug 31 01:04:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:04:56.961736 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebd880a-b719-4b9f-b87a-2a4a8d4f41b5 None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:57.000776 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3d33e5-ede4-492f-b7a8-8d82cf54c431 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:57.089964 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebd880a-b719-4b9f-b87a-2a4a8d4f41b5 None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:57.099963 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:57.105133 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3d33e5-ede4-492f-b7a8-8d82cf54c431 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:57.132711 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:57.138147 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40436c2d-25f5-457b-be1e-1c031693e862 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:57.139048 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40436c2d-25f5-457b-be1e-1c031693e862 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:57.139439 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40436c2d-25f5-457b-be1e-1c031693e862 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:57.139913 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40436c2d-25f5-457b-be1e-1c031693e862 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:57.140695 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40436c2d-25f5-457b-be1e-1c031693e862 None admin] RBAC: Adding query filter params (role.id=7d00a4cfa13744a69604ac2cf1a5816e, scope.project.id=60b2abc15c2849ba8a1ed92117cc981b, user.id=b3efb4f9daac4a0ca05a36dd361f17e5) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:04:57.145867 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40436c2d-25f5-457b-be1e-1c031693e862 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:57.145985 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64648115-0d67-4a1f-ab69-ffda3ff64d8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:57.145985 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64648115-0d67-4a1f-ab69-ffda3ff64d8a None admin] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:57.146189 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40436c2d-25f5-457b-be1e-1c031693e862 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:57.146287 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64648115-0d67-4a1f-ab69-ffda3ff64d8a None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:57.146287 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64648115-0d67-4a1f-ab69-ffda3ff64d8a None admin] PATH_INFO: `/v3/services` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:57.146287 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64648115-0d67-4a1f-ab69-ffda3ff64d8a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:57.146287 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64648115-0d67-4a1f-ab69-ffda3ff64d8a None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:57.156839 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 425/877] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Aug 31 01:04:56 2022] GET /identity/v3/role_assignments?user.id=b3efb4f9daac4a0ca05a36dd361f17e5&scope.project.id=60b2abc15c2849ba8a1ed92117cc981b&role.id=7d00a4cfa13744a69604ac2cf1a5816e => generated 608 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:04:57.234447 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40436c2d-25f5-457b-be1e-1c031693e862 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:57.235851 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 453/878] 10.222.0.20 () {60 vars in 1141 bytes} [Wed Aug 31 01:04:56 2022] POST /identity/v3/services => generated 269 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 01:04:57.358155 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40436c2d-25f5-457b-be1e-1c031693e862 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:57.402438 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d843561e-414c-43ea-a02b-4f5cac391cc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:57.403087 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d843561e-414c-43ea-a02b-4f5cac391cc5 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:57.403371 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d843561e-414c-43ea-a02b-4f5cac391cc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:57.403643 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d843561e-414c-43ea-a02b-4f5cac391cc5 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:57.404220 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d843561e-414c-43ea-a02b-4f5cac391cc5 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:57.405653 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d843561e-414c-43ea-a02b-4f5cac391cc5 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:57.415896 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application [None req-d843561e-414c-43ea-a02b-4f5cac391cc5 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 01:04:57.415896 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:04:57.415896 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 01:04:57.415896 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 01:04:57.415896 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 01:04:57.415896 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 01:04:57.415896 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 01:04:57.415896 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 01:04:57.415896 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 01:04:57.415896 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Aug 31 01:04:57.415896 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 01:04:57.415896 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Aug 31 01:04:57.415896 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:04:57.415896 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:04:57.415896 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:04:57.415896 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:04:57.415896 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:04:57.416952 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:04:57.416952 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:04:57.416952 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:04:57.416952 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:04:57.416952 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:04:57.416952 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:04:57.416952 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 01:04:57.416952 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 01:04:57.416952 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 01:04:57.416952 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 01:04:57.416952 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:04:57.416952 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:04:57.416952 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:04:57.416952 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:04:57.416952 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:04:57.416952 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:04:57.417817 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:04:57.417817 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application with Lock( Aug 31 01:04:57.417817 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:04:57.417817 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:04:57.417817 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:04:57.417817 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:04:57.417817 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:04:57.417817 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:04:57.417817 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:04:57.417817 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:04:57.417817 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 01:04:57.417817 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 01:04:57.417817 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 01:04:57.417817 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Aug 31 01:04:57.417817 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 426/879] 10.222.0.20 () {56 vars in 1126 bytes} [Wed Aug 31 01:04:57 2022] GET /identity/v3/services/object-store => generated 93 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 01:04:57.423101 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64648115-0d67-4a1f-ab69-ffda3ff64d8a None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:57.534827 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64648115-0d67-4a1f-ab69-ffda3ff64d8a None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:57.570258 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6fbfa4-405f-4bb1-a453-19394dca0c96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:57.571117 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6fbfa4-405f-4bb1-a453-19394dca0c96 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:57.571550 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6fbfa4-405f-4bb1-a453-19394dca0c96 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:57.571974 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6fbfa4-405f-4bb1-a453-19394dca0c96 None admin] PATH_INFO: `/v3/services` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:57.572775 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6fbfa4-405f-4bb1-a453-19394dca0c96 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:04:57.573267 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6fbfa4-405f-4bb1-a453-19394dca0c96 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:57.575040 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6fbfa4-405f-4bb1-a453-19394dca0c96 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:57.580570 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 454/880] 10.222.0.20 () {56 vars in 1122 bytes} [Wed Aug 31 01:04:57 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:04:57.586791 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d843561e-414c-43ea-a02b-4f5cac391cc5 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:57.619935 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:57.696391 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d843561e-414c-43ea-a02b-4f5cac391cc5 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:57.731741 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe549c0e-143b-4f8c-b290-7d5b2304b1b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:57.732361 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe549c0e-143b-4f8c-b290-7d5b2304b1b2 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:57.732641 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe549c0e-143b-4f8c-b290-7d5b2304b1b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:57.732942 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe549c0e-143b-4f8c-b290-7d5b2304b1b2 None admin] PATH_INFO: `/v3/services` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:57.733572 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe549c0e-143b-4f8c-b290-7d5b2304b1b2 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:04:57.733872 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe549c0e-143b-4f8c-b290-7d5b2304b1b2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:57.735252 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe549c0e-143b-4f8c-b290-7d5b2304b1b2 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:57.740257 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 427/881] 10.222.0.20 () {56 vars in 1122 bytes} [Wed Aug 31 01:04:57 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:04:58.441690 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bd6492-e282-4b67-82f9-869783d2f149 None None] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:58.442246 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bd6492-e282-4b67-82f9-869783d2f149 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:58.442508 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bd6492-e282-4b67-82f9-869783d2f149 None None] PATH_INFO: `/` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:58.444754 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 455/882] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:04:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:04:58.450224 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73b86bd-2fc0-4986-ae86-12e2e4402ef9 None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:58.450620 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73b86bd-2fc0-4986-ae86-12e2e4402ef9 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:58.451001 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73b86bd-2fc0-4986-ae86-12e2e4402ef9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:58.469794 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-c73b86bd-2fc0-4986-ae86-12e2e4402ef9 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:04:58.470958 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-c73b86bd-2fc0-4986-ae86-12e2e4402ef9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:04:58.542537 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 428/883] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:04:58 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 01:04:58.804509 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bd6492-e282-4b67-82f9-869783d2f149 None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:58.807302 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:58.918961 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bd6492-e282-4b67-82f9-869783d2f149 None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:58.953781 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e593e3-cba5-4e40-a226-316a989f46a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:58.954410 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e593e3-cba5-4e40-a226-316a989f46a3 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:58.954694 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e593e3-cba5-4e40-a226-316a989f46a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:58.954970 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e593e3-cba5-4e40-a226-316a989f46a3 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:58.955546 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e593e3-cba5-4e40-a226-316a989f46a3 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:58.956923 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e593e3-cba5-4e40-a226-316a989f46a3 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:58.965758 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application [None req-f7e593e3-cba5-4e40-a226-316a989f46a3 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 31 01:04:58.965758 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:04:58.965758 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 01:04:58.965758 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 01:04:58.965758 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 01:04:58.965758 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 01:04:58.965758 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 01:04:58.965758 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 01:04:58.965758 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 31 01:04:58.965758 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Aug 31 01:04:58.965758 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 01:04:58.965758 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Aug 31 01:04:58.965758 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:04:58.965758 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:04:58.965758 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:04:58.965758 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:04:58.965758 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:04:58.966725 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:04:58.966725 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:04:58.966725 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:04:58.966725 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:04:58.966725 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:04:58.966725 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:04:58.966725 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 01:04:58.966725 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 01:04:58.966725 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 01:04:58.966725 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 01:04:58.966725 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:04:58.966725 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:04:58.966725 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:04:58.966725 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:04:58.966725 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:04:58.966725 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:04:58.967563 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:04:58.967563 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application with Lock( Aug 31 01:04:58.967563 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:04:58.967563 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:04:58.967563 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:04:58.967563 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:04:58.967563 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:04:58.967563 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:04:58.967563 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:04:58.967563 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:04:58.967563 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 01:04:58.967563 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 01:04:58.967563 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 31 01:04:58.967563 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Aug 31 01:04:58.967563 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 456/884] 10.222.0.20 () {56 vars in 1114 bytes} [Wed Aug 31 01:04:58 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:04:58.972888 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73b86bd-2fc0-4986-ae86-12e2e4402ef9 None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:59.001052 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:59.079359 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73b86bd-2fc0-4986-ae86-12e2e4402ef9 None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:59.115136 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ce105f-0934-4ac3-8d04-5f6ae9a5a27e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:59.115851 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ce105f-0934-4ac3-8d04-5f6ae9a5a27e None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:59.116130 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ce105f-0934-4ac3-8d04-5f6ae9a5a27e None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:59.116402 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ce105f-0934-4ac3-8d04-5f6ae9a5a27e None admin] PATH_INFO: `/v3/services` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:59.117062 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ce105f-0934-4ac3-8d04-5f6ae9a5a27e None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:04:59.117364 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ce105f-0934-4ac3-8d04-5f6ae9a5a27e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:59.118735 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ce105f-0934-4ac3-8d04-5f6ae9a5a27e None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:59.128129 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 429/885] 10.222.0.20 () {56 vars in 1114 bytes} [Wed Aug 31 01:04:58 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:04:59.129513 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e593e3-cba5-4e40-a226-316a989f46a3 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:59.197747 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:59.234722 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e593e3-cba5-4e40-a226-316a989f46a3 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:59.238546 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4385927-5376-49fc-94d5-141e34445a38 None None] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:59.238832 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4385927-5376-49fc-94d5-141e34445a38 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:59.239107 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4385927-5376-49fc-94d5-141e34445a38 None None] PATH_INFO: `/` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:59.240549 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 430/886] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:04:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:04:59.242056 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedbe91a-4468-4120-8db1-9a1b07aac8f3 None None] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:59.242329 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedbe91a-4468-4120-8db1-9a1b07aac8f3 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:59.242599 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedbe91a-4468-4120-8db1-9a1b07aac8f3 None None] PATH_INFO: `/` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:59.244146 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 431/887] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:04:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:04:59.247334 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b75d01e-5128-427e-846a-cf5fe03b0d3e None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:59.247624 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b75d01e-5128-427e-846a-cf5fe03b0d3e None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:59.247900 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b75d01e-5128-427e-846a-cf5fe03b0d3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:59.267077 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-3b75d01e-5128-427e-846a-cf5fe03b0d3e None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:04:59.268569 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-3b75d01e-5128-427e-846a-cf5fe03b0d3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:04:59.273041 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4767985-d80b-40c6-8eb7-1372a74099cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:59.273651 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4767985-d80b-40c6-8eb7-1372a74099cb None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:59.273930 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4767985-d80b-40c6-8eb7-1372a74099cb None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:59.274199 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4767985-d80b-40c6-8eb7-1372a74099cb None admin] PATH_INFO: `/v3/services` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:59.274820 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4767985-d80b-40c6-8eb7-1372a74099cb None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:04:59.275120 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4767985-d80b-40c6-8eb7-1372a74099cb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:59.276475 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4767985-d80b-40c6-8eb7-1372a74099cb None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:59.284952 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 457/888] 10.222.0.20 () {56 vars in 1114 bytes} [Wed Aug 31 01:04:59 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:04:59.284952 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3029d96-0ed7-4ba8-9ada-f3b9554b1253 None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:59.284952 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3029d96-0ed7-4ba8-9ada-f3b9554b1253 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:59.284952 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3029d96-0ed7-4ba8-9ada-f3b9554b1253 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:59.301733 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-b3029d96-0ed7-4ba8-9ada-f3b9554b1253 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:04:59.302917 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-b3029d96-0ed7-4ba8-9ada-f3b9554b1253 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:04:59.342643 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 432/889] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:04:59 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 01:04:59.343446 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b75d01e-5128-427e-846a-cf5fe03b0d3e None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:59.367309 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:59.377975 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 458/890] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:04:59 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 01:04:59.455212 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b75d01e-5128-427e-846a-cf5fe03b0d3e None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:59.489851 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5866bd-fa96-444f-b278-ba306e4a13be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:59.491215 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5866bd-fa96-444f-b278-ba306e4a13be None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:59.491666 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5866bd-fa96-444f-b278-ba306e4a13be None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:59.492087 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5866bd-fa96-444f-b278-ba306e4a13be None admin] PATH_INFO: `/v3/endpoints` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:59.493023 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5866bd-fa96-444f-b278-ba306e4a13be None admin] RBAC: Adding query filter params (interface=public, service_id=990bf3bbd0004d51ab41eb6c4f46ffc3, region_id=RegionOne) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:04:59.493492 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5866bd-fa96-444f-b278-ba306e4a13be None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:59.495009 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5866bd-fa96-444f-b278-ba306e4a13be None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:59.502895 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 433/891] 10.222.0.20 () {56 vars in 1251 bytes} [Wed Aug 31 01:04:59 2022] GET /identity/v3/endpoints?service_id=990bf3bbd0004d51ab41eb6c4f46ffc3&interface=public®ion_id=RegionOne => generated 194 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:04:59.509101 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3029d96-0ed7-4ba8-9ada-f3b9554b1253 None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:59.571108 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:59.613761 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3029d96-0ed7-4ba8-9ada-f3b9554b1253 None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:59.625272 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5866bd-fa96-444f-b278-ba306e4a13be None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:59.650022 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39efeaad-061b-472b-99a3-5a214cce8f47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:59.650972 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39efeaad-061b-472b-99a3-5a214cce8f47 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:59.651466 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39efeaad-061b-472b-99a3-5a214cce8f47 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:59.651927 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39efeaad-061b-472b-99a3-5a214cce8f47 None admin] PATH_INFO: `/v3/services` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:59.652749 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39efeaad-061b-472b-99a3-5a214cce8f47 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:59.654398 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39efeaad-061b-472b-99a3-5a214cce8f47 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:59.660594 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 459/892] 10.222.0.20 () {56 vars in 1087 bytes} [Wed Aug 31 01:04:59 2022] GET /identity/v3/services => generated 1268 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:04:59.668752 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39efeaad-061b-472b-99a3-5a214cce8f47 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:59.724802 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:59.750272 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5866bd-fa96-444f-b278-ba306e4a13be None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:59.786923 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080b0577-2a4c-4b86-af68-0af2b9a3b729 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:59.788185 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080b0577-2a4c-4b86-af68-0af2b9a3b729 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:59.788379 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080b0577-2a4c-4b86-af68-0af2b9a3b729 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:59.788663 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080b0577-2a4c-4b86-af68-0af2b9a3b729 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:59.789326 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080b0577-2a4c-4b86-af68-0af2b9a3b729 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:59.790757 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080b0577-2a4c-4b86-af68-0af2b9a3b729 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:59.798059 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39efeaad-061b-472b-99a3-5a214cce8f47 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:59.800242 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application [None req-080b0577-2a4c-4b86-af68-0af2b9a3b729 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 31 01:04:59.800242 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:04:59.800242 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 01:04:59.800242 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 01:04:59.800242 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 01:04:59.800242 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 01:04:59.800242 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 01:04:59.800242 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 01:04:59.800242 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 31 01:04:59.800242 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Aug 31 01:04:59.800242 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 01:04:59.800242 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Aug 31 01:04:59.800242 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:04:59.800242 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:04:59.800242 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:04:59.800242 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:04:59.801209 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:04:59.801209 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:04:59.801209 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:04:59.801209 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:04:59.801209 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:04:59.801209 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:04:59.801209 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:04:59.801209 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 01:04:59.801209 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 01:04:59.801209 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 01:04:59.801209 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 01:04:59.801209 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:04:59.801209 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:04:59.801209 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:04:59.801209 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:04:59.801209 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:04:59.801209 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:04:59.802081 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:04:59.802081 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application with Lock( Aug 31 01:04:59.802081 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:04:59.802081 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:04:59.802081 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:04:59.802081 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:04:59.802081 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:04:59.802081 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:04:59.802081 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:04:59.802081 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:04:59.802081 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 01:04:59.802081 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 01:04:59.802081 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 31 01:04:59.802081 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Aug 31 01:04:59.802081 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 434/893] 10.222.0.20 () {56 vars in 1132 bytes} [Wed Aug 31 01:04:59 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:04:59.803994 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23448fe-7c0e-489d-8e35-4ac05066fa8e None None] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:59.803994 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23448fe-7c0e-489d-8e35-4ac05066fa8e None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:59.803994 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23448fe-7c0e-489d-8e35-4ac05066fa8e None None] PATH_INFO: `/` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:59.805885 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 435/894] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:04:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:04:59.810978 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3383822-6f96-4a31-a27f-1f562bf32835 None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:59.811988 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3383822-6f96-4a31-a27f-1f562bf32835 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:59.811988 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3383822-6f96-4a31-a27f-1f562bf32835 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:59.829873 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-c3383822-6f96-4a31-a27f-1f562bf32835 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:04:59.831333 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-c3383822-6f96-4a31-a27f-1f562bf32835 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:04:59.833864 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:59.845701 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b2f8e1-cbbc-4d21-96df-a4a8b6c57739 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:59.846337 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b2f8e1-cbbc-4d21-96df-a4a8b6c57739 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:59.846615 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b2f8e1-cbbc-4d21-96df-a4a8b6c57739 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:59.846884 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b2f8e1-cbbc-4d21-96df-a4a8b6c57739 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:59.857440 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b2f8e1-cbbc-4d21-96df-a4a8b6c57739 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 01:04:59.862526 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application [None req-09b2f8e1-cbbc-4d21-96df-a4a8b6c57739 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 31 01:04:59.862526 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:04:59.862526 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:04:59.862526 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:04:59.862526 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:04:59.862526 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:04:59.862526 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:04:59.862526 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:04:59.862526 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:04:59.862526 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:04:59.862526 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:04:59.862526 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:04:59.862526 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 01:04:59.862526 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 01:04:59.862526 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 01:04:59.862526 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 01:04:59.863640 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 01:04:59.863640 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 01:04:59.863640 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 01:04:59.863640 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 01:04:59.863640 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:04:59.863640 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:04:59.863640 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:04:59.863640 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:04:59.863640 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:04:59.863640 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:04:59.863640 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:04:59.863640 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application with Lock( Aug 31 01:04:59.863640 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:04:59.863640 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:04:59.863640 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:04:59.863640 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:04:59.864760 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:04:59.864760 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:04:59.864760 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:04:59.864760 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:04:59.864760 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 01:04:59.864760 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 01:04:59.864760 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 01:04:59.864760 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 01:04:59.864760 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 01:04:59.864760 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 01:04:59.864760 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 31 01:04:59.864760 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Aug 31 01:04:59.871825 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 460/895] 10.222.0.20 () {56 vars in 1120 bytes} [Wed Aug 31 01:04:59 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:04:59.872782 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b2f8e1-cbbc-4d21-96df-a4a8b6c57739 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:59.906672 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 436/896] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:04:59 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 01:04:59.917645 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3383822-6f96-4a31-a27f-1f562bf32835 None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:59.987895 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b2f8e1-cbbc-4d21-96df-a4a8b6c57739 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:59.998005 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:05:00.022097 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3383822-6f96-4a31-a27f-1f562bf32835 None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:05:00.026435 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24a98ac-930f-4c5c-b187-c827f53305c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:05:00.027327 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24a98ac-930f-4c5c-b187-c827f53305c9 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:00.027840 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24a98ac-930f-4c5c-b187-c827f53305c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:00.028311 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24a98ac-930f-4c5c-b187-c827f53305c9 None admin] PATH_INFO: `/v3/services` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:00.029178 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24a98ac-930f-4c5c-b187-c827f53305c9 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:05:00.029683 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24a98ac-930f-4c5c-b187-c827f53305c9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:05:00.031241 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24a98ac-930f-4c5c-b187-c827f53305c9 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:05:00.041074 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 461/897] 10.222.0.20 () {56 vars in 1126 bytes} [Wed Aug 31 01:04:59 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:05:00.042141 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24a98ac-930f-4c5c-b187-c827f53305c9 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:05:00.060081 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac338a2-d6ba-4cc1-95f3-4208265d87e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:05:00.060686 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac338a2-d6ba-4cc1-95f3-4208265d87e2 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:00.061006 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac338a2-d6ba-4cc1-95f3-4208265d87e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:00.061283 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac338a2-d6ba-4cc1-95f3-4208265d87e2 None admin] PATH_INFO: `/v3/roles` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:00.061948 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac338a2-d6ba-4cc1-95f3-4208265d87e2 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:05:00.062394 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac338a2-d6ba-4cc1-95f3-4208265d87e2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:05:00.063836 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac338a2-d6ba-4cc1-95f3-4208265d87e2 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:05:00.075283 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 437/898] 10.222.0.20 () {56 vars in 1115 bytes} [Wed Aug 31 01:04:59 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:05:00.080594 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac338a2-d6ba-4cc1-95f3-4208265d87e2 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:05:00.095837 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:05:00.156298 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24a98ac-930f-4c5c-b187-c827f53305c9 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:05:00.202333 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb2a030-16e8-49fb-aebd-e31248e3bd6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:05:00.203148 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb2a030-16e8-49fb-aebd-e31248e3bd6f None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:00.203648 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb2a030-16e8-49fb-aebd-e31248e3bd6f None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:00.203931 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb2a030-16e8-49fb-aebd-e31248e3bd6f None admin] PATH_INFO: `/v3/services` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:00.205408 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb2a030-16e8-49fb-aebd-e31248e3bd6f None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:05:00.205721 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb2a030-16e8-49fb-aebd-e31248e3bd6f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:05:00.207676 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb2a030-16e8-49fb-aebd-e31248e3bd6f None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:05:00.209965 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac338a2-d6ba-4cc1-95f3-4208265d87e2 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:05:00.214104 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 462/899] 10.222.0.20 () {56 vars in 1126 bytes} [Wed Aug 31 01:05:00 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:05:00.215174 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb2a030-16e8-49fb-aebd-e31248e3bd6f None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:05:00.247107 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd342210-4800-4ef9-839c-59fd1c666366 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:05:00.247732 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd342210-4800-4ef9-839c-59fd1c666366 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:00.248030 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd342210-4800-4ef9-839c-59fd1c666366 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:00.248281 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd342210-4800-4ef9-839c-59fd1c666366 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:00.257064 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:05:00.261541 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd342210-4800-4ef9-839c-59fd1c666366 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:05:00.262990 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd342210-4800-4ef9-839c-59fd1c666366 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:05:00.277515 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application [None req-fd342210-4800-4ef9-839c-59fd1c666366 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 01:05:00.277515 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:05:00.277515 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:05:00.277515 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:05:00.277515 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:05:00.277515 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:05:00.277515 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:05:00.277515 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:05:00.277515 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:05:00.277515 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:05:00.277515 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:05:00.277515 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:05:00.277515 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 01:05:00.277515 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 01:05:00.277515 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 01:05:00.277515 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 01:05:00.278683 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:05:00.278683 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:05:00.278683 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:05:00.278683 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:05:00.278683 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:05:00.278683 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:05:00.278683 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:05:00.278683 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application with Lock( Aug 31 01:05:00.278683 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:05:00.278683 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:05:00.278683 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:05:00.278683 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:05:00.278683 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:05:00.278683 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:05:00.278683 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:05:00.278683 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:05:00.278683 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 01:05:00.279843 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 01:05:00.279843 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 01:05:00.279843 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Aug 31 01:05:00.283653 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 438/900] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 01:05:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:05:00.284506 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd342210-4800-4ef9-839c-59fd1c666366 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:05:00.321888 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb2a030-16e8-49fb-aebd-e31248e3bd6f None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:05:00.335459 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:05:00.357212 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cbfef4-1ecd-4582-baf4-77a358dc570f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:05:00.357816 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cbfef4-1ecd-4582-baf4-77a358dc570f None admin] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:00.358094 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cbfef4-1ecd-4582-baf4-77a358dc570f None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:00.358405 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cbfef4-1ecd-4582-baf4-77a358dc570f None admin] PATH_INFO: `/v3/services` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:00.359156 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58cbfef4-1ecd-4582-baf4-77a358dc570f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:05:00.360763 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58cbfef4-1ecd-4582-baf4-77a358dc570f None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:05:00.393720 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd342210-4800-4ef9-839c-59fd1c666366 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:05:00.426885 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b77376a-af0e-40fb-a764-b9a722857508 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:05:00.427563 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b77376a-af0e-40fb-a764-b9a722857508 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:00.427853 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b77376a-af0e-40fb-a764-b9a722857508 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:00.428149 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b77376a-af0e-40fb-a764-b9a722857508 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:00.429417 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b77376a-af0e-40fb-a764-b9a722857508 None admin] RBAC: Adding query filter params (interface=public, service_id=4bdd59f116884087addc9da4a97c1b43, region_id=RegionOne) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:05:00.429730 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b77376a-af0e-40fb-a764-b9a722857508 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:05:00.431100 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b77376a-af0e-40fb-a764-b9a722857508 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:05:00.435998 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 439/901] 10.222.0.20 () {56 vars in 1251 bytes} [Wed Aug 31 01:05:00 2022] GET /identity/v3/endpoints?service_id=4bdd59f116884087addc9da4a97c1b43&interface=public®ion_id=RegionOne => generated 194 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:05:00.444749 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b77376a-af0e-40fb-a764-b9a722857508 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:05:00.444952 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 463/902] 10.222.0.20 () {60 vars in 1141 bytes} [Wed Aug 31 01:05:00 2022] POST /identity/v3/services => generated 241 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:05:00.445935 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cbfef4-1ecd-4582-baf4-77a358dc570f None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:05:00.537125 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:05:00.572537 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b77376a-af0e-40fb-a764-b9a722857508 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:05:00.587997 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cbfef4-1ecd-4582-baf4-77a358dc570f None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:05:00.622933 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3755813-594c-451c-ae32-7e21153cc250 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:05:00.623997 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3755813-594c-451c-ae32-7e21153cc250 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:00.624288 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3755813-594c-451c-ae32-7e21153cc250 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:00.624874 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3755813-594c-451c-ae32-7e21153cc250 None admin] PATH_INFO: `/v3/domains` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:00.625534 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3755813-594c-451c-ae32-7e21153cc250 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:05:00.625837 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3755813-594c-451c-ae32-7e21153cc250 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:05:00.627216 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3755813-594c-451c-ae32-7e21153cc250 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:05:00.635956 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb339ad-873e-4f36-96c3-66646d7783e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:05:00.636672 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb339ad-873e-4f36-96c3-66646d7783e8 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:00.636988 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb339ad-873e-4f36-96c3-66646d7783e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:00.637263 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb339ad-873e-4f36-96c3-66646d7783e8 None admin] PATH_INFO: `/v3/services` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:00.637880 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb339ad-873e-4f36-96c3-66646d7783e8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:05:00.639291 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb339ad-873e-4f36-96c3-66646d7783e8 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:05:00.640002 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 440/903] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 31 01:05:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:05:00.645705 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3755813-594c-451c-ae32-7e21153cc250 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:05:00.646144 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 464/904] 10.222.0.20 () {56 vars in 1087 bytes} [Wed Aug 31 01:05:00 2022] GET /identity/v3/services => generated 1497 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:05:00.761108 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3755813-594c-451c-ae32-7e21153cc250 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:05:00.777859 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:05:00.801371 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54bf8bb-23e7-4e64-bb8c-2649f2475746 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:05:00.802004 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54bf8bb-23e7-4e64-bb8c-2649f2475746 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:00.802280 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54bf8bb-23e7-4e64-bb8c-2649f2475746 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:00.802547 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54bf8bb-23e7-4e64-bb8c-2649f2475746 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:00.811230 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54bf8bb-23e7-4e64-bb8c-2649f2475746 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:05:00.813099 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54bf8bb-23e7-4e64-bb8c-2649f2475746 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:05:00.825956 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application [None req-c54bf8bb-23e7-4e64-bb8c-2649f2475746 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 31 01:05:00.825956 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:05:00.825956 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:05:00.825956 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:05:00.825956 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:05:00.825956 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:05:00.825956 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:05:00.825956 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:05:00.825956 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:05:00.825956 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:05:00.825956 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:05:00.825956 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:05:00.825956 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 01:05:00.825956 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 01:05:00.825956 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 01:05:00.825956 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 01:05:00.827153 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:05:00.827153 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:05:00.827153 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 01:05:00.827153 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 01:05:00.827153 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 01:05:00.827153 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 01:05:00.827153 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:05:00.827153 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:05:00.827153 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:05:00.827153 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:05:00.827153 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:05:00.827153 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application with Lock( Aug 31 01:05:00.827153 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:05:00.827153 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:05:00.827153 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:05:00.827153 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:05:00.827153 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:05:00.827153 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:05:00.828577 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:05:00.828577 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:05:00.828577 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 01:05:00.828577 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 01:05:00.828577 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 01:05:00.828577 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 01:05:00.828577 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 01:05:00.828577 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 01:05:00.828577 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 31 01:05:00.828577 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Aug 31 01:05:00.828577 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 441/905] 10.222.0.20 () {56 vars in 1117 bytes} [Wed Aug 31 01:05:00 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:05:00.836763 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb339ad-873e-4f36-96c3-66646d7783e8 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:05:00.872287 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:05:00.946242 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb339ad-873e-4f36-96c3-66646d7783e8 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:05:00.982361 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb370b0-c5fa-4f17-8ea6-04cde883ca19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:05:00.982974 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb370b0-c5fa-4f17-8ea6-04cde883ca19 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:00.983259 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb370b0-c5fa-4f17-8ea6-04cde883ca19 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:00.983538 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb370b0-c5fa-4f17-8ea6-04cde883ca19 None admin] PATH_INFO: `/v3/users` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:00.984272 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb370b0-c5fa-4f17-8ea6-04cde883ca19 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:05:00.984584 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb370b0-c5fa-4f17-8ea6-04cde883ca19 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:05:00.986046 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb370b0-c5fa-4f17-8ea6-04cde883ca19 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:05:01.004545 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 465/906] 10.222.0.20 () {56 vars in 1149 bytes} [Wed Aug 31 01:05:00 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:05:01.010675 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54bf8bb-23e7-4e64-bb8c-2649f2475746 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:05:01.119429 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54bf8bb-23e7-4e64-bb8c-2649f2475746 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:05:01.154641 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21db7bc-7b81-498d-b333-056d7fa72937 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:05:01.155248 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21db7bc-7b81-498d-b333-056d7fa72937 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:01.155526 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21db7bc-7b81-498d-b333-056d7fa72937 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:01.155794 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21db7bc-7b81-498d-b333-056d7fa72937 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:01.205259 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:05:01.232361 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21db7bc-7b81-498d-b333-056d7fa72937 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:05:01.234090 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21db7bc-7b81-498d-b333-056d7fa72937 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:05:01.338616 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 442/907] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Aug 31 01:05:01 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 01:05:01.344267 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb370b0-c5fa-4f17-8ea6-04cde883ca19 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:05:01.436639 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:05:01.446611 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb370b0-c5fa-4f17-8ea6-04cde883ca19 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:05:01.483026 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b18898-67b0-4bef-b934-8c799b4413be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:05:01.483026 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b18898-67b0-4bef-b934-8c799b4413be None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:01.483026 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b18898-67b0-4bef-b934-8c799b4413be None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:01.483026 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b18898-67b0-4bef-b934-8c799b4413be None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:01.494266 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b18898-67b0-4bef-b934-8c799b4413be None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:05:01.495725 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b18898-67b0-4bef-b934-8c799b4413be None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:05:01.512208 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application [None req-45b18898-67b0-4bef-b934-8c799b4413be None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 01:05:01.512208 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:05:01.512208 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:05:01.512208 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:05:01.512208 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:05:01.512208 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:05:01.512208 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:05:01.512208 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:05:01.512208 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:05:01.512208 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:05:01.512208 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:05:01.512208 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:05:01.512208 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 01:05:01.512208 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 01:05:01.512208 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 01:05:01.512208 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 01:05:01.513435 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:05:01.513435 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:05:01.513435 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:05:01.513435 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:05:01.513435 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:05:01.513435 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:05:01.513435 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:05:01.513435 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application with Lock( Aug 31 01:05:01.513435 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:05:01.513435 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:05:01.513435 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:05:01.513435 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:05:01.513435 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:05:01.513435 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:05:01.513435 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:05:01.513435 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:05:01.513435 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 01:05:01.514522 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 01:05:01.514522 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 01:05:01.514522 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Aug 31 01:05:01.514522 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 466/908] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 01:05:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:05:01.519464 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21db7bc-7b81-498d-b333-056d7fa72937 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:05:01.595296 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:05:01.603716 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b47499-693b-4e1f-9f0c-b56cd68a6f0e None None] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:01.604195 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b47499-693b-4e1f-9f0c-b56cd68a6f0e None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:01.604291 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b47499-693b-4e1f-9f0c-b56cd68a6f0e None None] PATH_INFO: `/` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:01.605906 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 467/909] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:05:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:05:01.613846 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71207092-f15c-4149-860e-ee3712528e1c None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:01.620867 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71207092-f15c-4149-860e-ee3712528e1c None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:01.620867 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71207092-f15c-4149-860e-ee3712528e1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:01.628789 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21db7bc-7b81-498d-b333-056d7fa72937 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:05:01.639415 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-71207092-f15c-4149-860e-ee3712528e1c None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:05:01.640842 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-71207092-f15c-4149-860e-ee3712528e1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:05:01.665839 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ef4e4e-989d-4b41-b8d2-f8c1cb510b3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:05:01.666697 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ef4e4e-989d-4b41-b8d2-f8c1cb510b3e None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:01.667147 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ef4e4e-989d-4b41-b8d2-f8c1cb510b3e None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:01.667574 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ef4e4e-989d-4b41-b8d2-f8c1cb510b3e None admin] PATH_INFO: `/v3/domains` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:01.668360 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ef4e4e-989d-4b41-b8d2-f8c1cb510b3e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:05:01.668847 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ef4e4e-989d-4b41-b8d2-f8c1cb510b3e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:05:01.670370 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ef4e4e-989d-4b41-b8d2-f8c1cb510b3e None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:05:01.682988 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 443/910] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 31 01:05:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:05:01.688571 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ef4e4e-989d-4b41-b8d2-f8c1cb510b3e None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:05:01.730745 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 468/911] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:05:01 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 01:05:01.807152 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ef4e4e-989d-4b41-b8d2-f8c1cb510b3e None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:05:01.843438 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af567f2b-c6af-4709-a4da-7c9fe62959b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:05:01.844094 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af567f2b-c6af-4709-a4da-7c9fe62959b7 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:01.844374 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af567f2b-c6af-4709-a4da-7c9fe62959b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:01.844651 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af567f2b-c6af-4709-a4da-7c9fe62959b7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:01.846964 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:05:01.853707 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af567f2b-c6af-4709-a4da-7c9fe62959b7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:05:01.855200 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af567f2b-c6af-4709-a4da-7c9fe62959b7 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:05:01.867257 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application [None req-af567f2b-c6af-4709-a4da-7c9fe62959b7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 01:05:01.867257 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:05:01.867257 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:05:01.867257 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:05:01.867257 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:05:01.867257 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:05:01.867257 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:05:01.867257 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:05:01.867257 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:05:01.867257 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:05:01.867257 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:05:01.867257 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:05:01.867257 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 01:05:01.867257 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 01:05:01.867257 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 01:05:01.873044 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 01:05:01.873044 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:05:01.873044 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:05:01.873044 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:05:01.873044 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:05:01.873044 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:05:01.873044 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:05:01.873044 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:05:01.873044 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application with Lock( Aug 31 01:05:01.873044 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:05:01.873044 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:05:01.873044 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:05:01.873044 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:05:01.873044 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:05:01.873044 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:05:01.873044 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:05:01.873044 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:05:01.874014 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 01:05:01.874014 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 01:05:01.874014 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 01:05:01.874014 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 01:05:01.874014 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 01:05:01.874014 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 01:05:01.874014 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 01:05:01.874014 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Aug 31 01:05:01.874014 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 444/912] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Aug 31 01:05:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 01:05:01.877871 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71207092-f15c-4149-860e-ee3712528e1c None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:05:01.955465 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:05:01.986407 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71207092-f15c-4149-860e-ee3712528e1c None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:05:02.019065 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af567f2b-c6af-4709-a4da-7c9fe62959b7 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:05:02.023391 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db897804-40c7-40df-9172-67992d283e91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:05:02.024258 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db897804-40c7-40df-9172-67992d283e91 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:02.024763 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db897804-40c7-40df-9172-67992d283e91 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:02.025324 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db897804-40c7-40df-9172-67992d283e91 None admin] PATH_INFO: `/v3/projects` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:02.026246 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db897804-40c7-40df-9172-67992d283e91 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:05:02.026766 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db897804-40c7-40df-9172-67992d283e91 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:05:02.028427 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db897804-40c7-40df-9172-67992d283e91 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:05:02.041346 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 469/913] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Aug 31 01:05:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:05:02.056849 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db897804-40c7-40df-9172-67992d283e91 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:05:02.126323 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af567f2b-c6af-4709-a4da-7c9fe62959b7 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:05:02.161229 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db897804-40c7-40df-9172-67992d283e91 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:05:02.161694 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab64df7-5ea8-4da9-9f82-de558b30cf45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:05:02.162338 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab64df7-5ea8-4da9-9f82-de558b30cf45 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:02.163150 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab64df7-5ea8-4da9-9f82-de558b30cf45 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:02.163423 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab64df7-5ea8-4da9-9f82-de558b30cf45 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:02.164008 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab64df7-5ea8-4da9-9f82-de558b30cf45 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:05:02.166614 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab64df7-5ea8-4da9-9f82-de558b30cf45 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:05:02.177895 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application [None req-bab64df7-5ea8-4da9-9f82-de558b30cf45 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 31 01:05:02.177895 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:05:02.177895 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 01:05:02.177895 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 01:05:02.177895 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 01:05:02.177895 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 01:05:02.177895 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 01:05:02.177895 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 01:05:02.177895 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 31 01:05:02.177895 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Aug 31 01:05:02.177895 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 01:05:02.177895 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Aug 31 01:05:02.177895 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:05:02.177895 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:05:02.177895 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:05:02.177895 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:05:02.177895 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:05:02.179539 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:05:02.179539 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:05:02.179539 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:05:02.179539 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:05:02.179539 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:05:02.179539 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:05:02.179539 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 01:05:02.179539 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 01:05:02.179539 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 01:05:02.179539 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 01:05:02.179539 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:05:02.179539 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:05:02.179539 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:05:02.179539 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:05:02.179539 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:05:02.179539 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:05:02.180440 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:05:02.180440 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application with Lock( Aug 31 01:05:02.180440 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:05:02.180440 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:05:02.180440 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:05:02.180440 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:05:02.180440 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:05:02.180440 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:05:02.180440 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:05:02.180440 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:05:02.180440 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 01:05:02.180440 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 01:05:02.180440 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 31 01:05:02.180440 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Aug 31 01:05:02.185059 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 445/914] 10.222.0.20 () {56 vars in 1114 bytes} [Wed Aug 31 01:05:02 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:05:02.185960 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab64df7-5ea8-4da9-9f82-de558b30cf45 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:05:02.202144 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:05:02.207597 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60696e2d-9bc0-4937-bdae-4f953d07fe52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:05:02.208521 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60696e2d-9bc0-4937-bdae-4f953d07fe52 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:02.209069 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60696e2d-9bc0-4937-bdae-4f953d07fe52 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:02.210167 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60696e2d-9bc0-4937-bdae-4f953d07fe52 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:02.211289 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60696e2d-9bc0-4937-bdae-4f953d07fe52 None admin] RBAC: Adding query filter params (role.id=586b679002e645439ed44fe1ca4a7cc6, scope.project.id=60b2abc15c2849ba8a1ed92117cc981b, user.id=b3efb4f9daac4a0ca05a36dd361f17e5) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:05:02.211828 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60696e2d-9bc0-4937-bdae-4f953d07fe52 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:05:02.213566 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60696e2d-9bc0-4937-bdae-4f953d07fe52 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:05:02.224842 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 470/915] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Aug 31 01:05:02 2022] GET /identity/v3/role_assignments?user.id=b3efb4f9daac4a0ca05a36dd361f17e5&scope.project.id=60b2abc15c2849ba8a1ed92117cc981b&role.id=586b679002e645439ed44fe1ca4a7cc6 => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 01:05:02.258469 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:05:02.306475 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab64df7-5ea8-4da9-9f82-de558b30cf45 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:05:02.344158 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f0d0d6-a318-4e3b-b4ac-6ba872268d20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:05:02.344813 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f0d0d6-a318-4e3b-b4ac-6ba872268d20 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:02.345099 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f0d0d6-a318-4e3b-b4ac-6ba872268d20 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:02.345376 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f0d0d6-a318-4e3b-b4ac-6ba872268d20 None admin] PATH_INFO: `/v3/services` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:02.346012 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f0d0d6-a318-4e3b-b4ac-6ba872268d20 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:05:02.346317 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f0d0d6-a318-4e3b-b4ac-6ba872268d20 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:05:02.347715 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f0d0d6-a318-4e3b-b4ac-6ba872268d20 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:05:02.354026 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 446/916] 10.222.0.20 () {56 vars in 1114 bytes} [Wed Aug 31 01:05:02 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:05:02.359762 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60696e2d-9bc0-4937-bdae-4f953d07fe52 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:05:02.469506 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60696e2d-9bc0-4937-bdae-4f953d07fe52 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:05:02.505476 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766025c4-9dcb-4574-9f06-e79e9da64038 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:05:02.505713 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766025c4-9dcb-4574-9f06-e79e9da64038 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:02.507125 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766025c4-9dcb-4574-9f06-e79e9da64038 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:02.507125 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766025c4-9dcb-4574-9f06-e79e9da64038 None admin] PATH_INFO: `/v3/services` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:02.507125 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766025c4-9dcb-4574-9f06-e79e9da64038 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:05:02.507358 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766025c4-9dcb-4574-9f06-e79e9da64038 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:05:02.508540 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766025c4-9dcb-4574-9f06-e79e9da64038 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:05:02.514064 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 471/917] 10.222.0.20 () {56 vars in 1114 bytes} [Wed Aug 31 01:05:02 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:05:02.519702 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f0d0d6-a318-4e3b-b4ac-6ba872268d20 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:05:02.570369 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6585cb85-6de3-4a29-8923-35a3f81a8290 None None] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:02.570796 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6585cb85-6de3-4a29-8923-35a3f81a8290 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:02.571078 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6585cb85-6de3-4a29-8923-35a3f81a8290 None None] PATH_INFO: `/` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:02.573087 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 472/918] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:05:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 01:05:02.577630 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d9777a-4394-4e45-ad84-b3111f7f1581 None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:02.577918 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d9777a-4394-4e45-ad84-b3111f7f1581 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:02.578202 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d9777a-4394-4e45-ad84-b3111f7f1581 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:02.600975 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-a8d9777a-4394-4e45-ad84-b3111f7f1581 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:05:02.602520 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-a8d9777a-4394-4e45-ad84-b3111f7f1581 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:05:02.643562 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f0d0d6-a318-4e3b-b4ac-6ba872268d20 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:05:02.666202 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:05:02.667437 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:05:02.679488 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd161a8-ec9f-4081-aee1-4e224bdc39ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:05:02.685151 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd161a8-ec9f-4081-aee1-4e224bdc39ac None admin] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:02.685151 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd161a8-ec9f-4081-aee1-4e224bdc39ac None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:02.685151 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd161a8-ec9f-4081-aee1-4e224bdc39ac None admin] PATH_INFO: `/v3/endpoints` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:02.685151 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd161a8-ec9f-4081-aee1-4e224bdc39ac None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:05:02.685151 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd161a8-ec9f-4081-aee1-4e224bdc39ac None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:05:02.685970 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 473/919] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:05:02 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 01:05:02.706395 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ceda0e-1b54-4a89-9c0a-a05b5e4789d3 None None] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:02.706677 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ceda0e-1b54-4a89-9c0a-a05b5e4789d3 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:02.707011 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ceda0e-1b54-4a89-9c0a-a05b5e4789d3 None None] PATH_INFO: `/` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:02.708585 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 474/920] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:05:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 01:05:02.713697 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a77d0a-7a7d-4a11-a4f3-842ec5465e32 None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:02.713977 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a77d0a-7a7d-4a11-a4f3-842ec5465e32 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:02.714262 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a77d0a-7a7d-4a11-a4f3-842ec5465e32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:02.732302 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-c7a77d0a-7a7d-4a11-a4f3-842ec5465e32 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:05:02.733456 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-c7a77d0a-7a7d-4a11-a4f3-842ec5465e32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:05:02.767818 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 447/921] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 01:05:02 2022] POST /identity/v3/endpoints => generated 346 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:05:02.836774 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 475/922] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:05:02 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 01:05:02.978656 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd161a8-ec9f-4081-aee1-4e224bdc39ac None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:05:03.083861 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd161a8-ec9f-4081-aee1-4e224bdc39ac None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:05:03.113904 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a77d0a-7a7d-4a11-a4f3-842ec5465e32 None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:05:03.119479 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bebb09-5103-4c72-bb45-8c4ba23b78a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:05:03.120350 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bebb09-5103-4c72-bb45-8c4ba23b78a0 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:03.120851 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bebb09-5103-4c72-bb45-8c4ba23b78a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:03.121322 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bebb09-5103-4c72-bb45-8c4ba23b78a0 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:03.122082 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bebb09-5103-4c72-bb45-8c4ba23b78a0 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:05:03.123630 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bebb09-5103-4c72-bb45-8c4ba23b78a0 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:05:03.133733 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application [None req-c8bebb09-5103-4c72-bb45-8c4ba23b78a0 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 01:05:03.133733 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:05:03.133733 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 01:05:03.133733 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 01:05:03.133733 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 01:05:03.133733 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 01:05:03.133733 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 01:05:03.133733 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 01:05:03.133733 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 01:05:03.133733 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Aug 31 01:05:03.133733 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 01:05:03.133733 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Aug 31 01:05:03.133733 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:05:03.133733 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:05:03.133733 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:05:03.133733 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:05:03.133733 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:05:03.134966 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:05:03.134966 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:05:03.134966 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:05:03.134966 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:05:03.134966 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:05:03.134966 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:05:03.134966 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 01:05:03.134966 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 01:05:03.134966 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 01:05:03.134966 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 01:05:03.134966 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:05:03.134966 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:05:03.134966 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:05:03.134966 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:05:03.134966 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:05:03.134966 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:05:03.135989 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:05:03.135989 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application with Lock( Aug 31 01:05:03.135989 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:05:03.135989 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:05:03.135989 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:05:03.135989 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:05:03.135989 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:05:03.135989 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:05:03.135989 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:05:03.135989 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:05:03.135989 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 01:05:03.135989 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 01:05:03.135989 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 01:05:03.135989 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Aug 31 01:05:03.138479 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 448/923] 10.222.0.20 () {56 vars in 1126 bytes} [Wed Aug 31 01:05:02 2022] GET /identity/v3/services/object-store => generated 93 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:05:03.144573 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bebb09-5103-4c72-bb45-8c4ba23b78a0 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:05:03.203090 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:05:03.226554 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a77d0a-7a7d-4a11-a4f3-842ec5465e32 None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:05:03.248984 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:05:03.264070 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bebb09-5103-4c72-bb45-8c4ba23b78a0 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:05:03.272872 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2149e2-0873-44b2-8ff0-117c27b0e56b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:05:03.274204 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2149e2-0873-44b2-8ff0-117c27b0e56b None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:03.274711 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2149e2-0873-44b2-8ff0-117c27b0e56b None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:03.275528 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2149e2-0873-44b2-8ff0-117c27b0e56b None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:03.276619 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2149e2-0873-44b2-8ff0-117c27b0e56b None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:05:03.278636 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2149e2-0873-44b2-8ff0-117c27b0e56b None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:05:03.290287 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application [None req-bd2149e2-0873-44b2-8ff0-117c27b0e56b None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 31 01:05:03.290287 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:05:03.290287 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 01:05:03.290287 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 01:05:03.290287 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 01:05:03.290287 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 01:05:03.290287 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 01:05:03.290287 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 01:05:03.290287 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 31 01:05:03.290287 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Aug 31 01:05:03.290287 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 01:05:03.290287 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Aug 31 01:05:03.290287 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:05:03.290287 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:05:03.290287 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:05:03.290287 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:05:03.291277 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:05:03.291277 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:05:03.291277 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:05:03.291277 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:05:03.291277 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:05:03.291277 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:05:03.291277 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:05:03.291277 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 01:05:03.291277 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 01:05:03.291277 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 01:05:03.291277 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 01:05:03.291277 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:05:03.291277 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:05:03.291277 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:05:03.291277 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:05:03.291277 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:05:03.291277 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:05:03.292153 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:05:03.292153 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application with Lock( Aug 31 01:05:03.292153 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:05:03.292153 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:05:03.292153 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:05:03.292153 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:05:03.292153 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:05:03.292153 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:05:03.292153 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:05:03.292153 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:05:03.292153 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 01:05:03.292153 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 01:05:03.292153 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 31 01:05:03.292153 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Aug 31 01:05:03.295028 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 476/924] 10.222.0.20 () {56 vars in 1132 bytes} [Wed Aug 31 01:05:03 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 01:05:03.301834 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2149e2-0873-44b2-8ff0-117c27b0e56b None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:05:03.311673 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249b6900-d348-4d05-af3a-18c2142d3625 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:05:03.312589 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249b6900-d348-4d05-af3a-18c2142d3625 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:03.313125 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249b6900-d348-4d05-af3a-18c2142d3625 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:03.313705 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249b6900-d348-4d05-af3a-18c2142d3625 None admin] PATH_INFO: `/v3/services` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:03.314417 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249b6900-d348-4d05-af3a-18c2142d3625 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:05:03.314795 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249b6900-d348-4d05-af3a-18c2142d3625 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:05:03.316395 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249b6900-d348-4d05-af3a-18c2142d3625 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:05:03.323061 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 449/925] 10.222.0.20 () {56 vars in 1122 bytes} [Wed Aug 31 01:05:03 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 01:05:03.329115 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249b6900-d348-4d05-af3a-18c2142d3625 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:05:03.435660 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2149e2-0873-44b2-8ff0-117c27b0e56b None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:05:03.438905 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249b6900-d348-4d05-af3a-18c2142d3625 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:05:03.446077 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:05:03.476544 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764ed1cc-4863-4016-b73a-6be9e8d32695 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:05:03.477510 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764ed1cc-4863-4016-b73a-6be9e8d32695 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:03.477878 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164f1596-3e0a-4eda-9217-9718c68f50c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:05:03.478119 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764ed1cc-4863-4016-b73a-6be9e8d32695 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:03.478481 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164f1596-3e0a-4eda-9217-9718c68f50c2 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:03.478740 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764ed1cc-4863-4016-b73a-6be9e8d32695 None admin] PATH_INFO: `/v3/services` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:03.478859 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164f1596-3e0a-4eda-9217-9718c68f50c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:03.479014 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164f1596-3e0a-4eda-9217-9718c68f50c2 None admin] PATH_INFO: `/v3/services` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:03.479636 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164f1596-3e0a-4eda-9217-9718c68f50c2 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:05:03.479937 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164f1596-3e0a-4eda-9217-9718c68f50c2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:05:03.480123 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764ed1cc-4863-4016-b73a-6be9e8d32695 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:05:03.480572 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764ed1cc-4863-4016-b73a-6be9e8d32695 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:05:03.481385 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164f1596-3e0a-4eda-9217-9718c68f50c2 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:05:03.482252 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764ed1cc-4863-4016-b73a-6be9e8d32695 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:05:03.492322 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 477/926] 10.222.0.20 () {56 vars in 1126 bytes} [Wed Aug 31 01:05:03 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:05:03.492817 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 450/927] 10.222.0.20 () {56 vars in 1122 bytes} [Wed Aug 31 01:05:03 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:05:03.493725 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164f1596-3e0a-4eda-9217-9718c68f50c2 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:05:03.494511 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764ed1cc-4863-4016-b73a-6be9e8d32695 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:05:03.549702 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:05:03.600931 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764ed1cc-4863-4016-b73a-6be9e8d32695 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:05:03.619750 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164f1596-3e0a-4eda-9217-9718c68f50c2 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:05:03.643124 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9185b6f-6f64-4717-9a46-2235fdaf6ed8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:05:03.643729 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9185b6f-6f64-4717-9a46-2235fdaf6ed8 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:03.644271 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9185b6f-6f64-4717-9a46-2235fdaf6ed8 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:03.644271 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9185b6f-6f64-4717-9a46-2235fdaf6ed8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:03.645236 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9185b6f-6f64-4717-9a46-2235fdaf6ed8 None admin] RBAC: Adding query filter params (interface=public, service_id=81a6706394f14a96aecda72ded4f638c, region_id=RegionOne) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:05:03.645541 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9185b6f-6f64-4717-9a46-2235fdaf6ed8 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:05:03.647651 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9185b6f-6f64-4717-9a46-2235fdaf6ed8 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:05:03.659355 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 451/928] 10.222.0.20 () {56 vars in 1251 bytes} [Wed Aug 31 01:05:03 2022] GET /identity/v3/endpoints?service_id=81a6706394f14a96aecda72ded4f638c&interface=public®ion_id=RegionOne => generated 194 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 01:05:03.660609 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9185b6f-6f64-4717-9a46-2235fdaf6ed8 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:05:03.663778 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc44395-9c5c-43d7-ba0a-924e5fbfb0ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:05:03.664392 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc44395-9c5c-43d7-ba0a-924e5fbfb0ca None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:03.664662 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc44395-9c5c-43d7-ba0a-924e5fbfb0ca None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:03.664966 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc44395-9c5c-43d7-ba0a-924e5fbfb0ca None admin] PATH_INFO: `/v3/services` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:03.665889 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc44395-9c5c-43d7-ba0a-924e5fbfb0ca None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:05:03.666192 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc44395-9c5c-43d7-ba0a-924e5fbfb0ca None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:05:03.668425 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc44395-9c5c-43d7-ba0a-924e5fbfb0ca None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:05:03.676808 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 478/929] 10.222.0.20 () {56 vars in 1126 bytes} [Wed Aug 31 01:05:03 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:05:03.682358 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc44395-9c5c-43d7-ba0a-924e5fbfb0ca None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:05:03.705176 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:05:03.779548 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9185b6f-6f64-4717-9a46-2235fdaf6ed8 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:05:03.819993 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc44395-9c5c-43d7-ba0a-924e5fbfb0ca None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:05:03.823609 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbf294e-2b26-481a-9dbd-70d2ab186263 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:05:03.824440 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbf294e-2b26-481a-9dbd-70d2ab186263 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:03.825165 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbf294e-2b26-481a-9dbd-70d2ab186263 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:03.825654 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbf294e-2b26-481a-9dbd-70d2ab186263 None admin] PATH_INFO: `/v3/services` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:03.826443 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbf294e-2b26-481a-9dbd-70d2ab186263 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:05:03.828594 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbf294e-2b26-481a-9dbd-70d2ab186263 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:05:03.830495 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:05:03.845076 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 452/930] 10.222.0.20 () {56 vars in 1087 bytes} [Wed Aug 31 01:05:03 2022] GET /identity/v3/services => generated 1497 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 01:05:03.869155 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff6006e-2808-4412-9921-12bebf750846 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:05:03.869786 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff6006e-2808-4412-9921-12bebf750846 None admin] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:03.870067 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff6006e-2808-4412-9921-12bebf750846 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:03.870341 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff6006e-2808-4412-9921-12bebf750846 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:03.871028 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff6006e-2808-4412-9921-12bebf750846 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:05:03.872508 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff6006e-2808-4412-9921-12bebf750846 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:05:03.991155 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 479/931] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 01:05:03 2022] POST /identity/v3/endpoints => generated 347 bytes in 310 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:05:04.166786 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d5a6b4-7b99-468d-a6d2-7ea212f760fa None None] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:04.167289 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d5a6b4-7b99-468d-a6d2-7ea212f760fa None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:04.167680 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d5a6b4-7b99-468d-a6d2-7ea212f760fa None None] PATH_INFO: `/` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:04.169193 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 453/932] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:05:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:05:04.174657 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f140a4f-e42f-40c7-bece-4685a11a71d0 None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:04.174950 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f140a4f-e42f-40c7-bece-4685a11a71d0 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:04.175224 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f140a4f-e42f-40c7-bece-4685a11a71d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:04.194662 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-4f140a4f-e42f-40c7-bece-4685a11a71d0 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:05:04.195869 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-4f140a4f-e42f-40c7-bece-4685a11a71d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:05:04.274277 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 480/933] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:05:04 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 01:05:04.546953 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d5a6b4-7b99-468d-a6d2-7ea212f760fa None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:05:04.649689 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:05:04.655522 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d5a6b4-7b99-468d-a6d2-7ea212f760fa None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:05:04.689916 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd16e84-859a-4d4c-8102-d3617cd76ec0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:05:04.690610 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd16e84-859a-4d4c-8102-d3617cd76ec0 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:04.691110 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd16e84-859a-4d4c-8102-d3617cd76ec0 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:04.691605 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd16e84-859a-4d4c-8102-d3617cd76ec0 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:04.703187 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd16e84-859a-4d4c-8102-d3617cd76ec0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 01:05:04.708134 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application [None req-4cd16e84-859a-4d4c-8102-d3617cd76ec0 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 31 01:05:04.708134 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:05:04.708134 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:05:04.708134 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:05:04.708134 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:05:04.708134 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:05:04.708134 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:05:04.708134 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:05:04.708134 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:05:04.708134 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:05:04.708134 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:05:04.708134 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:05:04.708134 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 01:05:04.708134 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 01:05:04.708134 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 01:05:04.708134 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 01:05:04.709108 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 01:05:04.709108 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 01:05:04.709108 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 01:05:04.709108 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 01:05:04.709108 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:05:04.709108 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:05:04.709108 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:05:04.709108 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:05:04.709108 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:05:04.709108 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:05:04.709108 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:05:04.709108 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application with Lock( Aug 31 01:05:04.709108 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:05:04.709108 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:05:04.709108 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:05:04.709108 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:05:04.710017 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:05:04.710017 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:05:04.710017 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:05:04.710017 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:05:04.710017 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 01:05:04.710017 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 01:05:04.710017 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 01:05:04.710017 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 01:05:04.710017 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 01:05:04.710017 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 01:05:04.710017 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 31 01:05:04.710017 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Aug 31 01:05:04.710017 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 454/934] 10.222.0.20 () {56 vars in 1120 bytes} [Wed Aug 31 01:05:04 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:05:04.715590 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f140a4f-e42f-40c7-bece-4685a11a71d0 None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:05:04.772227 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bcfab0-f6cd-453d-b2fe-2f65c4ed3060 None None] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:04.772578 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bcfab0-f6cd-453d-b2fe-2f65c4ed3060 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:04.772806 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bcfab0-f6cd-453d-b2fe-2f65c4ed3060 None None] PATH_INFO: `/` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:04.774281 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 455/935] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:05:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:05:04.779572 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310c8532-0c9e-4d76-9185-c4c7fd7f070a None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:04.780071 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310c8532-0c9e-4d76-9185-c4c7fd7f070a None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:04.780609 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310c8532-0c9e-4d76-9185-c4c7fd7f070a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:04.799473 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-310c8532-0c9e-4d76-9185-c4c7fd7f070a None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:05:04.801054 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-310c8532-0c9e-4d76-9185-c4c7fd7f070a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:05:04.817671 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:05:04.829725 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f140a4f-e42f-40c7-bece-4685a11a71d0 None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:05:04.873630 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61890649-42d4-47f3-bc54-9dbd86cace85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:05:04.874679 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61890649-42d4-47f3-bc54-9dbd86cace85 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:04.874960 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61890649-42d4-47f3-bc54-9dbd86cace85 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:04.875492 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61890649-42d4-47f3-bc54-9dbd86cace85 None admin] PATH_INFO: `/v3/roles` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:04.876161 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61890649-42d4-47f3-bc54-9dbd86cace85 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:05:04.876824 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61890649-42d4-47f3-bc54-9dbd86cace85 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:05:04.878890 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61890649-42d4-47f3-bc54-9dbd86cace85 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:05:04.885580 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 456/936] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:05:04 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 01:05:04.890600 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 481/937] 10.222.0.20 () {56 vars in 1115 bytes} [Wed Aug 31 01:05:04 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:05:04.900742 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310c8532-0c9e-4d76-9185-c4c7fd7f070a None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:05:04.993653 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:05:05.005962 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310c8532-0c9e-4d76-9185-c4c7fd7f070a None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:05:05.044528 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a41263-525c-4a47-85b9-b3134edd2345 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:05:05.045184 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a41263-525c-4a47-85b9-b3134edd2345 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:05.045464 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a41263-525c-4a47-85b9-b3134edd2345 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:05.045739 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a41263-525c-4a47-85b9-b3134edd2345 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:05.056854 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a41263-525c-4a47-85b9-b3134edd2345 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:05:05.058250 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a41263-525c-4a47-85b9-b3134edd2345 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:05:05.073265 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application [None req-89a41263-525c-4a47-85b9-b3134edd2345 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 01:05:05.073265 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:05:05.073265 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:05:05.073265 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:05:05.073265 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:05:05.073265 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:05:05.073265 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:05:05.073265 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:05:05.073265 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:05:05.073265 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:05:05.073265 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:05:05.073265 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:05:05.073265 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 01:05:05.073265 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 01:05:05.073265 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 01:05:05.073265 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 01:05:05.074225 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:05:05.074225 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:05:05.074225 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:05:05.074225 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:05:05.074225 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:05:05.074225 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:05:05.074225 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:05:05.074225 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application with Lock( Aug 31 01:05:05.074225 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:05:05.074225 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:05:05.074225 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:05:05.074225 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:05:05.074225 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:05:05.074225 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:05:05.074225 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:05:05.074225 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:05:05.074225 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 01:05:05.075372 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 01:05:05.075372 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 01:05:05.075372 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Aug 31 01:05:05.075372 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 457/938] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 01:05:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:05:05.081737 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61890649-42d4-47f3-bc54-9dbd86cace85 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:05:05.165771 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a41263-525c-4a47-85b9-b3134edd2345 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:05:05.194573 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61890649-42d4-47f3-bc54-9dbd86cace85 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:05:05.244252 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41854c29-7e25-4262-9046-923a0247e2b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:05:05.245320 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41854c29-7e25-4262-9046-923a0247e2b8 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:05.245985 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41854c29-7e25-4262-9046-923a0247e2b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:05.246466 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41854c29-7e25-4262-9046-923a0247e2b8 None admin] PATH_INFO: `/v3/domains` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:05.247324 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41854c29-7e25-4262-9046-923a0247e2b8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:05:05.247854 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41854c29-7e25-4262-9046-923a0247e2b8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:05:05.249543 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41854c29-7e25-4262-9046-923a0247e2b8 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:05:05.262608 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 482/939] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 31 01:05:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 01:05:05.267750 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41854c29-7e25-4262-9046-923a0247e2b8 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:05:05.284399 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a41263-525c-4a47-85b9-b3134edd2345 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:05:05.289814 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:05:05.329988 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0535a39b-65d0-4e82-9192-8df8b6c345d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:05:05.331081 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0535a39b-65d0-4e82-9192-8df8b6c345d7 None admin] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:05.331521 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0535a39b-65d0-4e82-9192-8df8b6c345d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:05.332366 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0535a39b-65d0-4e82-9192-8df8b6c345d7 None admin] PATH_INFO: `/v3/projects` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:05.333511 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0535a39b-65d0-4e82-9192-8df8b6c345d7 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:05:05.336272 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0535a39b-65d0-4e82-9192-8df8b6c345d7 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:05:05.339127 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: WARNING keystone.server.flask.common [None req-0535a39b-65d0-4e82-9192-8df8b6c345d7 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 31 01:05:05.354690 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:05:05.390166 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.sql.core [None req-0535a39b-65d0-4e82-9192-8df8b6c345d7 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 31 01:05:05.390166 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 31 01:05:05.390166 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [parameters: {'id': '9a3f5710674e4b0ab177b3fd652baa1b', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 31 01:05:05.390166 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81749) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 31 01:05:05.395724 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41854c29-7e25-4262-9046-923a0247e2b8 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:05:05.400622 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application [None req-0535a39b-65d0-4e82-9192-8df8b6c345d7 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 9a3f5710674e4b0ab177b3fd652baa1b.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 9a3f5710674e4b0ab177b3fd652baa1b. Aug 31 01:05:05.400622 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:05:05.400622 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 31 01:05:05.400622 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 31 01:05:05.400622 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 31 01:05:05.400622 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 31 01:05:05.400622 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 31 01:05:05.400622 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application result = self._query(query) Aug 31 01:05:05.400622 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 31 01:05:05.400622 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application conn.query(q) Aug 31 01:05:05.400622 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 31 01:05:05.400622 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 31 01:05:05.400622 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 31 01:05:05.401528 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application result.read() Aug 31 01:05:05.401528 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 31 01:05:05.401528 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 31 01:05:05.401528 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 31 01:05:05.401528 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 31 01:05:05.401528 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 31 01:05:05.401528 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 31 01:05:05.401528 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 31 01:05:05.401528 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 31 01:05:05.401528 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 31 01:05:05.401528 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Aug 31 01:05:05.401528 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 31 01:05:05.401528 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Aug 31 01:05:05.401528 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:05:05.401528 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 31 01:05:05.401528 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 31 01:05:05.401528 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Aug 31 01:05:05.401528 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return project_ref.to_dict() Aug 31 01:05:05.402483 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 31 01:05:05.402483 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application next(self.gen) Aug 31 01:05:05.402483 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 31 01:05:05.402483 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application yield resource Aug 31 01:05:05.402483 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 31 01:05:05.402483 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application next(self.gen) Aug 31 01:05:05.402483 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 31 01:05:05.402483 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application self.session.rollback() Aug 31 01:05:05.402483 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 31 01:05:05.402483 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application self.force_reraise() Aug 31 01:05:05.402483 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 31 01:05:05.402483 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application raise self.value Aug 31 01:05:05.402483 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 31 01:05:05.402483 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 31 01:05:05.402483 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 31 01:05:05.402483 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application session.commit() Aug 31 01:05:05.402483 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 31 01:05:05.403378 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 31 01:05:05.403378 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 31 01:05:05.403378 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application self._prepare_impl() Aug 31 01:05:05.403378 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 31 01:05:05.403378 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application self.session.flush() Aug 31 01:05:05.403378 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 31 01:05:05.403378 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application self._flush(objects) Aug 31 01:05:05.403378 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 31 01:05:05.403378 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 31 01:05:05.403378 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 31 01:05:05.403378 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application compat.raise_( Aug 31 01:05:05.403378 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 31 01:05:05.403378 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application raise exception Aug 31 01:05:05.403378 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 31 01:05:05.403378 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application flush_context.execute() Aug 31 01:05:05.403378 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 31 01:05:05.403378 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application rec.execute(self) Aug 31 01:05:05.404876 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 31 01:05:05.404876 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 31 01:05:05.404876 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 31 01:05:05.404876 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 31 01:05:05.404876 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 31 01:05:05.404876 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 31 01:05:05.404876 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 31 01:05:05.404876 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 31 01:05:05.404876 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 31 01:05:05.404876 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 31 01:05:05.404876 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 31 01:05:05.404876 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 31 01:05:05.404876 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 31 01:05:05.404876 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 31 01:05:05.404876 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 31 01:05:05.406195 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 31 01:05:05.406195 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 31 01:05:05.406195 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application raise exception Aug 31 01:05:05.406195 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 31 01:05:05.406195 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 31 01:05:05.406195 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 31 01:05:05.406195 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 31 01:05:05.406195 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 31 01:05:05.406195 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application result = self._query(query) Aug 31 01:05:05.406195 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 31 01:05:05.406195 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application conn.query(q) Aug 31 01:05:05.406195 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 31 01:05:05.406195 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 31 01:05:05.406195 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 31 01:05:05.406195 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application result.read() Aug 31 01:05:05.406195 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 31 01:05:05.406195 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 31 01:05:05.408051 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 31 01:05:05.408051 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 31 01:05:05.408051 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 31 01:05:05.408051 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 31 01:05:05.408051 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 31 01:05:05.408051 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 31 01:05:05.408051 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 31 01:05:05.408051 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 31 01:05:05.408051 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application [parameters: {'id': '9a3f5710674e4b0ab177b3fd652baa1b', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 31 01:05:05.408051 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 31 01:05:05.408051 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Aug 31 01:05:05.408051 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 01:05:05.408051 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Aug 31 01:05:05.408051 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:05:05.408051 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Aug 31 01:05:05.410254 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Aug 31 01:05:05.410254 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 31 01:05:05.410254 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 31 01:05:05.410254 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Aug 31 01:05:05.410254 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Aug 31 01:05:05.410254 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 01:05:05.410254 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Aug 31 01:05:05.410254 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:05:05.410254 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:05:05.410254 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:05:05.410254 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:05:05.410254 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:05:05.410254 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:05:05.410254 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:05:05.410254 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:05:05.410254 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:05:05.410254 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:05:05.413454 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:05:05.413454 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Aug 31 01:05:05.413454 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Aug 31 01:05:05.413454 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:05:05.413454 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:05:05.413454 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Aug 31 01:05:05.413454 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application raise exception.Conflict( Aug 31 01:05:05.413454 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 9a3f5710674e4b0ab177b3fd652baa1b. Aug 31 01:05:05.413454 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Aug 31 01:05:05.417145 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 458/940] 10.222.0.20 () {60 vars in 1140 bytes} [Wed Aug 31 01:05:05 2022] POST /identity/v3/projects => generated 262 bytes in 252 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:05:05.419120 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0535a39b-65d0-4e82-9192-8df8b6c345d7 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:05:05.441673 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d703d4-18a4-4fa3-8f6a-09be255aac2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:05:05.442450 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d703d4-18a4-4fa3-8f6a-09be255aac2f None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:05.442872 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d703d4-18a4-4fa3-8f6a-09be255aac2f None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:05.443253 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d703d4-18a4-4fa3-8f6a-09be255aac2f None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:05.451609 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d703d4-18a4-4fa3-8f6a-09be255aac2f None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:05:05.453593 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d703d4-18a4-4fa3-8f6a-09be255aac2f None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:05:05.468232 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application [None req-32d703d4-18a4-4fa3-8f6a-09be255aac2f None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 31 01:05:05.468232 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:05:05.468232 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:05:05.468232 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:05:05.468232 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:05:05.468232 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:05:05.468232 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:05:05.468232 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:05:05.468232 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:05:05.468232 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:05:05.468232 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:05:05.468232 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:05:05.468232 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 01:05:05.468232 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 01:05:05.468232 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 01:05:05.468232 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 01:05:05.470002 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:05:05.470002 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:05:05.470002 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 01:05:05.470002 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 01:05:05.470002 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 01:05:05.470002 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 01:05:05.470002 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:05:05.470002 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:05:05.470002 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:05:05.470002 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:05:05.470002 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:05:05.470002 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application with Lock( Aug 31 01:05:05.470002 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:05:05.470002 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:05:05.470002 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:05:05.470002 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:05:05.470002 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:05:05.470002 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:05:05.471267 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:05:05.471267 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:05:05.471267 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 01:05:05.471267 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 01:05:05.471267 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 01:05:05.471267 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 01:05:05.471267 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 01:05:05.471267 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 01:05:05.471267 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 31 01:05:05.471267 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Aug 31 01:05:05.471267 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 483/941] 10.222.0.20 () {56 vars in 1117 bytes} [Wed Aug 31 01:05:05 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:05:05.477688 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d703d4-18a4-4fa3-8f6a-09be255aac2f None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:05:05.557607 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:05:05.558433 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0535a39b-65d0-4e82-9192-8df8b6c345d7 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:05:05.600626 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ff4953-999b-4682-8433-38a5dd644116 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:05:05.602958 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d703d4-18a4-4fa3-8f6a-09be255aac2f None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:05:05.603087 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ff4953-999b-4682-8433-38a5dd644116 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:05.603373 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ff4953-999b-4682-8433-38a5dd644116 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:05.604111 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ff4953-999b-4682-8433-38a5dd644116 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:05.612061 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ff4953-999b-4682-8433-38a5dd644116 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:05:05.613755 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ff4953-999b-4682-8433-38a5dd644116 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:05:05.626178 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application [None req-a4ff4953-999b-4682-8433-38a5dd644116 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 01:05:05.626178 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:05:05.626178 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:05:05.626178 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:05:05.626178 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:05:05.626178 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:05:05.626178 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:05:05.626178 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:05:05.626178 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:05:05.626178 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:05:05.626178 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:05:05.626178 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:05:05.626178 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 01:05:05.626178 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 01:05:05.626178 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 01:05:05.627112 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 01:05:05.627112 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:05:05.627112 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:05:05.627112 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:05:05.627112 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:05:05.627112 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:05:05.627112 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:05:05.627112 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:05:05.627112 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application with Lock( Aug 31 01:05:05.627112 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:05:05.627112 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:05:05.627112 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:05:05.627112 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:05:05.627112 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:05:05.627112 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:05:05.627112 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:05:05.627112 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:05:05.628029 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 01:05:05.628029 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 01:05:05.628029 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 01:05:05.628029 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 01:05:05.628029 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 01:05:05.628029 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 01:05:05.628029 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 01:05:05.628029 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Aug 31 01:05:05.628029 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 459/942] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Aug 31 01:05:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:05:05.645963 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ff4953-999b-4682-8433-38a5dd644116 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:05:05.654142 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827c0bf2-5623-4e6b-91d7-2efc65a39aef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:05:05.654995 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827c0bf2-5623-4e6b-91d7-2efc65a39aef None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:05.655478 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827c0bf2-5623-4e6b-91d7-2efc65a39aef None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:05.655901 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827c0bf2-5623-4e6b-91d7-2efc65a39aef None admin] PATH_INFO: `/v3/users` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:05.656815 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827c0bf2-5623-4e6b-91d7-2efc65a39aef None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:05:05.657297 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827c0bf2-5623-4e6b-91d7-2efc65a39aef None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:05:05.658947 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827c0bf2-5623-4e6b-91d7-2efc65a39aef None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:05:05.682430 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 484/943] 10.222.0.20 () {56 vars in 1149 bytes} [Wed Aug 31 01:05:05 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:05:05.687919 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827c0bf2-5623-4e6b-91d7-2efc65a39aef None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:05:05.763376 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:05:05.841043 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ff4953-999b-4682-8433-38a5dd644116 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:05:05.859886 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827c0bf2-5623-4e6b-91d7-2efc65a39aef None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:05:05.893074 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:05:05.898635 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7904f3e-d819-4e65-a399-e3261b431af0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:05:05.899353 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7904f3e-d819-4e65-a399-e3261b431af0 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:05.899635 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7904f3e-d819-4e65-a399-e3261b431af0 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:05.899908 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7904f3e-d819-4e65-a399-e3261b431af0 None admin] PATH_INFO: `/v3/projects` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:05.900578 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7904f3e-d819-4e65-a399-e3261b431af0 None admin] RBAC: Adding query filter params (name=service) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:05:05.901005 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7904f3e-d819-4e65-a399-e3261b431af0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:05:05.901476 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5ffb17-28e6-4661-a4d4-ee1c72bd110d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:05:05.902212 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5ffb17-28e6-4661-a4d4-ee1c72bd110d None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:05.902449 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7904f3e-d819-4e65-a399-e3261b431af0 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:05:05.902548 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5ffb17-28e6-4661-a4d4-ee1c72bd110d None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:05.902753 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5ffb17-28e6-4661-a4d4-ee1c72bd110d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:05.912905 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5ffb17-28e6-4661-a4d4-ee1c72bd110d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:05:05.914355 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5ffb17-28e6-4661-a4d4-ee1c72bd110d None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:05:05.938256 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application [None req-af5ffb17-28e6-4661-a4d4-ee1c72bd110d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 01:05:05.938256 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:05:05.938256 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:05:05.938256 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:05:05.938256 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:05:05.938256 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:05:05.938256 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:05:05.938256 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:05:05.938256 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:05:05.938256 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:05:05.938256 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:05:05.938256 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:05:05.938256 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 01:05:05.938256 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 01:05:05.938256 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 01:05:05.938256 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 01:05:05.939282 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:05:05.939282 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:05:05.939282 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:05:05.939282 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:05:05.939282 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:05:05.939282 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:05:05.939282 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:05:05.939282 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application with Lock( Aug 31 01:05:05.939282 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:05:05.939282 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:05:05.939282 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:05:05.939282 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:05:05.939282 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:05:05.939282 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:05:05.939282 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:05:05.939282 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:05:05.939282 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 01:05:05.940174 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 01:05:05.940174 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 01:05:05.940174 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Aug 31 01:05:05.942514 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 485/944] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 01:05:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:05:05.942877 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 460/945] 10.222.0.20 () {56 vars in 1112 bytes} [Wed Aug 31 01:05:05 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 01:05:05.944935 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffebb17e-0933-48c2-a136-892042b832c6 None None] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:05.945503 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffebb17e-0933-48c2-a136-892042b832c6 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:05.946000 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffebb17e-0933-48c2-a136-892042b832c6 None None] PATH_INFO: `/` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:05.952168 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7904f3e-d819-4e65-a399-e3261b431af0 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:05:05.952894 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 486/946] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:05:05 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:05:05.954738 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385a865a-5b2a-4915-b609-061ac3e3408c None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:05.955256 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385a865a-5b2a-4915-b609-061ac3e3408c None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:05.955748 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385a865a-5b2a-4915-b609-061ac3e3408c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:05.995261 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-385a865a-5b2a-4915-b609-061ac3e3408c None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:05:05.998697 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-385a865a-5b2a-4915-b609-061ac3e3408c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:05:06.119923 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 487/947] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:05:05 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 01:05:06.120985 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5964254e-4a75-4d4b-a793-192e7d66624b None None] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:06.121276 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5964254e-4a75-4d4b-a793-192e7d66624b None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:06.121551 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5964254e-4a75-4d4b-a793-192e7d66624b None None] PATH_INFO: `/` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:06.122780 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 488/948] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:05:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:05:06.128584 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ea8a4e-84c8-4c0a-b92d-6cc80565fc8c None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:06.128897 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ea8a4e-84c8-4c0a-b92d-6cc80565fc8c None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:06.129174 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ea8a4e-84c8-4c0a-b92d-6cc80565fc8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:06.138213 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7904f3e-d819-4e65-a399-e3261b431af0 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:05:06.147596 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-b5ea8a4e-84c8-4c0a-b92d-6cc80565fc8c None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:05:06.148829 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-b5ea8a4e-84c8-4c0a-b92d-6cc80565fc8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:05:06.159806 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:05:06.182121 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f4a78d-e4d4-4fb6-b3eb-0d06f552fb3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:05:06.182827 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f4a78d-e4d4-4fb6-b3eb-0d06f552fb3d None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:06.183410 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f4a78d-e4d4-4fb6-b3eb-0d06f552fb3d None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:06.183410 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f4a78d-e4d4-4fb6-b3eb-0d06f552fb3d None admin] PATH_INFO: `/v3/domains` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:06.184018 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f4a78d-e4d4-4fb6-b3eb-0d06f552fb3d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:05:06.184324 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f4a78d-e4d4-4fb6-b3eb-0d06f552fb3d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:05:06.185731 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f4a78d-e4d4-4fb6-b3eb-0d06f552fb3d None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:05:06.202364 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 461/949] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 31 01:05:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:05:06.207820 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f4a78d-e4d4-4fb6-b3eb-0d06f552fb3d None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:05:06.234261 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 489/950] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:05:06 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 01:05:06.320931 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f4a78d-e4d4-4fb6-b3eb-0d06f552fb3d None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:05:06.360264 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91269919-9311-48f1-a7e9-7ae65999a4f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:05:06.360925 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91269919-9311-48f1-a7e9-7ae65999a4f4 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:06.361206 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91269919-9311-48f1-a7e9-7ae65999a4f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:06.361657 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91269919-9311-48f1-a7e9-7ae65999a4f4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:06.378799 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91269919-9311-48f1-a7e9-7ae65999a4f4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:05:06.380210 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91269919-9311-48f1-a7e9-7ae65999a4f4 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:05:06.392412 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application [None req-91269919-9311-48f1-a7e9-7ae65999a4f4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 01:05:06.392412 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:05:06.392412 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:05:06.392412 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:05:06.392412 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:05:06.392412 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:05:06.392412 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:05:06.392412 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:05:06.392412 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:05:06.392412 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:05:06.392412 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:05:06.392412 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:05:06.392412 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 01:05:06.392412 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 01:05:06.392412 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 01:05:06.393487 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 01:05:06.393487 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:05:06.393487 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:05:06.393487 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:05:06.393487 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:05:06.393487 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:05:06.393487 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:05:06.393487 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:05:06.393487 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application with Lock( Aug 31 01:05:06.393487 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:05:06.393487 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:05:06.393487 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:05:06.393487 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:05:06.393487 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:05:06.393487 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:05:06.393487 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:05:06.393487 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:05:06.394593 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 01:05:06.394593 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 01:05:06.394593 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 01:05:06.394593 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 01:05:06.394593 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 01:05:06.394593 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 01:05:06.394593 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 01:05:06.394593 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Aug 31 01:05:06.394593 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 462/951] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Aug 31 01:05:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 01:05:06.399680 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ea8a4e-84c8-4c0a-b92d-6cc80565fc8c None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:05:06.399816 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91269919-9311-48f1-a7e9-7ae65999a4f4 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:05:06.436121 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:05:06.450174 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:05:06.505385 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91269919-9311-48f1-a7e9-7ae65999a4f4 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:05:06.526235 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ea8a4e-84c8-4c0a-b92d-6cc80565fc8c None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:05:06.542380 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e6ab3b-d0ed-4ee8-9c64-9e5527d5cab8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:05:06.543001 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e6ab3b-d0ed-4ee8-9c64-9e5527d5cab8 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:06.543368 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e6ab3b-d0ed-4ee8-9c64-9e5527d5cab8 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:06.543580 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e6ab3b-d0ed-4ee8-9c64-9e5527d5cab8 None admin] PATH_INFO: `/v3/projects` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:06.544257 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e6ab3b-d0ed-4ee8-9c64-9e5527d5cab8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:05:06.544566 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e6ab3b-d0ed-4ee8-9c64-9e5527d5cab8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:05:06.546043 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e6ab3b-d0ed-4ee8-9c64-9e5527d5cab8 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:05:06.563899 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 463/952] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Aug 31 01:05:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 01:05:06.563899 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e6ab3b-d0ed-4ee8-9c64-9e5527d5cab8 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:05:06.573247 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff0c230-295c-413b-89b4-776768b75147 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:05:06.573885 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff0c230-295c-413b-89b4-776768b75147 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:06.574171 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff0c230-295c-413b-89b4-776768b75147 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:06.574448 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff0c230-295c-413b-89b4-776768b75147 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:06.575367 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff0c230-295c-413b-89b4-776768b75147 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:05:06.576665 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff0c230-295c-413b-89b4-776768b75147 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:05:06.585647 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application [None req-1ff0c230-295c-413b-89b4-776768b75147 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 01:05:06.585647 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:05:06.585647 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 01:05:06.585647 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 01:05:06.585647 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 01:05:06.585647 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 01:05:06.585647 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 01:05:06.585647 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 01:05:06.585647 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 01:05:06.585647 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Aug 31 01:05:06.585647 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 01:05:06.585647 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Aug 31 01:05:06.585647 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:05:06.585647 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:05:06.585647 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:05:06.585647 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:05:06.585647 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:05:06.586954 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:05:06.586954 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:05:06.586954 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:05:06.586954 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:05:06.586954 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:05:06.586954 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:05:06.586954 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 01:05:06.586954 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 01:05:06.586954 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 01:05:06.586954 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 01:05:06.586954 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:05:06.586954 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:05:06.586954 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:05:06.586954 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:05:06.586954 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:05:06.586954 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:05:06.588049 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:05:06.588049 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application with Lock( Aug 31 01:05:06.588049 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:05:06.588049 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:05:06.588049 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:05:06.588049 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:05:06.588049 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:05:06.588049 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:05:06.588049 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:05:06.588049 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:05:06.588049 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 01:05:06.588049 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 01:05:06.588049 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 01:05:06.588049 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Aug 31 01:05:06.588049 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 490/953] 10.222.0.20 () {56 vars in 1126 bytes} [Wed Aug 31 01:05:06 2022] GET /identity/v3/services/object-store => generated 93 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:05:06.589126 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff0c230-295c-413b-89b4-776768b75147 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:05:06.667879 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:05:06.674341 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e6ab3b-d0ed-4ee8-9c64-9e5527d5cab8 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:05:06.693442 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff0c230-295c-413b-89b4-776768b75147 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:05:06.700772 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:05:06.711897 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9452aa7-dfe4-4519-bfbb-5c843720ab95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:05:06.712519 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9452aa7-dfe4-4519-bfbb-5c843720ab95 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:06.712839 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9452aa7-dfe4-4519-bfbb-5c843720ab95 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:06.713117 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9452aa7-dfe4-4519-bfbb-5c843720ab95 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:06.713706 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9452aa7-dfe4-4519-bfbb-5c843720ab95 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:05:06.715071 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9452aa7-dfe4-4519-bfbb-5c843720ab95 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:05:06.724137 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application [None req-e9452aa7-dfe4-4519-bfbb-5c843720ab95 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 31 01:05:06.724137 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:05:06.724137 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 01:05:06.724137 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 01:05:06.724137 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 01:05:06.724137 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 01:05:06.724137 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 01:05:06.724137 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 01:05:06.724137 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 31 01:05:06.724137 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Aug 31 01:05:06.724137 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 01:05:06.724137 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Aug 31 01:05:06.724137 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:05:06.724137 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:05:06.724137 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:05:06.724137 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:05:06.724137 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:05:06.725819 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:05:06.725819 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:05:06.725819 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:05:06.725819 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:05:06.725819 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:05:06.725819 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:05:06.725819 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 01:05:06.725819 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 01:05:06.725819 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 01:05:06.725819 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 01:05:06.725819 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:05:06.725819 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:05:06.725819 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:05:06.725819 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:05:06.725819 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:05:06.725819 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:05:06.729865 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:05:06.729865 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application with Lock( Aug 31 01:05:06.729865 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:05:06.729865 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:05:06.729865 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:05:06.729865 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:05:06.729865 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:05:06.729865 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:05:06.729865 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:05:06.729865 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:05:06.729865 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 01:05:06.729865 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 01:05:06.729865 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 31 01:05:06.729865 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Aug 31 01:05:06.729865 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 464/954] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Aug 31 01:05:06 2022] GET /identity/v3/services/compute => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:05:06.730678 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9452aa7-dfe4-4519-bfbb-5c843720ab95 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:05:06.732877 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ec9a12-f112-4051-b167-df5079055878 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:05:06.733884 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ec9a12-f112-4051-b167-df5079055878 None admin] REQUEST_METHOD: `PUT` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:06.734481 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ec9a12-f112-4051-b167-df5079055878 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:06.734755 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ec9a12-f112-4051-b167-df5079055878 None admin] PATH_INFO: `/v3/projects/60b2abc15c2849ba8a1ed92117cc981b/users/b3efb4f9daac4a0ca05a36dd361f17e5/roles/586b679002e645439ed44fe1ca4a7cc6` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:06.737016 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ec9a12-f112-4051-b167-df5079055878 None admin] RBAC: Authorizing `identity:create_grant(project_id=60b2abc15c2849ba8a1ed92117cc981b, user_id=b3efb4f9daac4a0ca05a36dd361f17e5, role_id=586b679002e645439ed44fe1ca4a7cc6)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:05:06.738450 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ec9a12-f112-4051-b167-df5079055878 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:05:06.821015 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 491/955] 10.222.0.20 () {58 vars in 1439 bytes} [Wed Aug 31 01:05:06 2022] PUT /identity/v3/projects/60b2abc15c2849ba8a1ed92117cc981b/users/b3efb4f9daac4a0ca05a36dd361f17e5/roles/586b679002e645439ed44fe1ca4a7cc6 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:05:06.822058 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ec9a12-f112-4051-b167-df5079055878 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:05:06.832176 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9452aa7-dfe4-4519-bfbb-5c843720ab95 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:05:06.883828 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115ce073-470f-467b-9a7a-d1d098b0bde1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:05:06.885238 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115ce073-470f-467b-9a7a-d1d098b0bde1 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:06.885679 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115ce073-470f-467b-9a7a-d1d098b0bde1 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:06.886477 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115ce073-470f-467b-9a7a-d1d098b0bde1 None admin] PATH_INFO: `/v3/services` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:06.887743 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115ce073-470f-467b-9a7a-d1d098b0bde1 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:05:06.888270 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115ce073-470f-467b-9a7a-d1d098b0bde1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:05:06.890602 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115ce073-470f-467b-9a7a-d1d098b0bde1 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:05:06.899358 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 465/956] 10.222.0.20 () {56 vars in 1122 bytes} [Wed Aug 31 01:05:06 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:05:06.912975 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115ce073-470f-467b-9a7a-d1d098b0bde1 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:05:06.935666 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:05:06.978065 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ec9a12-f112-4051-b167-df5079055878 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:05:07.018406 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf12c63-a532-48b2-a118-57c2cb179bf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:05:07.019024 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf12c63-a532-48b2-a118-57c2cb179bf1 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:07.019306 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf12c63-a532-48b2-a118-57c2cb179bf1 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:07.019581 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf12c63-a532-48b2-a118-57c2cb179bf1 None admin] PATH_INFO: `/v3/services` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:07.020371 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf12c63-a532-48b2-a118-57c2cb179bf1 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:05:07.020675 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf12c63-a532-48b2-a118-57c2cb179bf1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:05:07.023208 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf12c63-a532-48b2-a118-57c2cb179bf1 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:05:07.027703 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115ce073-470f-467b-9a7a-d1d098b0bde1 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:05:07.029096 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 492/957] 10.222.0.20 () {56 vars in 1112 bytes} [Wed Aug 31 01:05:06 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:05:07.038247 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf12c63-a532-48b2-a118-57c2cb179bf1 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:05:07.066862 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d245c2e-d0e7-4a4f-af73-5b4a12e9db0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:05:07.067727 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d245c2e-d0e7-4a4f-af73-5b4a12e9db0b None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:07.068188 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d245c2e-d0e7-4a4f-af73-5b4a12e9db0b None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:07.068647 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d245c2e-d0e7-4a4f-af73-5b4a12e9db0b None admin] PATH_INFO: `/v3/services` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:07.069465 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d245c2e-d0e7-4a4f-af73-5b4a12e9db0b None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:05:07.069916 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d245c2e-d0e7-4a4f-af73-5b4a12e9db0b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:05:07.071459 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d245c2e-d0e7-4a4f-af73-5b4a12e9db0b None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:05:07.075965 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:05:07.083677 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 466/958] 10.222.0.20 () {56 vars in 1122 bytes} [Wed Aug 31 01:05:06 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:05:07.085730 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d245c2e-d0e7-4a4f-af73-5b4a12e9db0b None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:05:07.168130 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf12c63-a532-48b2-a118-57c2cb179bf1 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:05:07.181087 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:05:07.197584 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d245c2e-d0e7-4a4f-af73-5b4a12e9db0b None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:05:07.220811 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a166e81-985a-452b-a653-716970065480 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:05:07.221808 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a166e81-985a-452b-a653-716970065480 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:07.222240 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a166e81-985a-452b-a653-716970065480 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:07.222819 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a166e81-985a-452b-a653-716970065480 None admin] PATH_INFO: `/v3/services` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:07.223925 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a166e81-985a-452b-a653-716970065480 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:05:07.224659 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a166e81-985a-452b-a653-716970065480 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:05:07.226765 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a166e81-985a-452b-a653-716970065480 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:05:07.232986 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 493/959] 10.222.0.20 () {56 vars in 1112 bytes} [Wed Aug 31 01:05:07 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:05:07.235263 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826ebc8c-e7f8-462c-a61c-2b18421f4d3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:05:07.235977 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826ebc8c-e7f8-462c-a61c-2b18421f4d3d None admin] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:07.236331 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826ebc8c-e7f8-462c-a61c-2b18421f4d3d None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:07.236677 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826ebc8c-e7f8-462c-a61c-2b18421f4d3d None admin] PATH_INFO: `/v3/endpoints` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:07.237542 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826ebc8c-e7f8-462c-a61c-2b18421f4d3d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:05:07.239032 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826ebc8c-e7f8-462c-a61c-2b18421f4d3d None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:05:07.322784 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 467/960] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 01:05:07 2022] POST /identity/v3/endpoints => generated 349 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 01:05:08.072295 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b5261b-a473-4a49-997c-35f9ad6b5f6c None None] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:08.072829 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b5261b-a473-4a49-997c-35f9ad6b5f6c None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:08.072909 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b5261b-a473-4a49-997c-35f9ad6b5f6c None None] PATH_INFO: `/` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:08.074622 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 494/961] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:05:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:05:08.080160 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b379ed3-ef13-4d3c-9c57-8a3c2b1a51c8 None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:08.080590 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b379ed3-ef13-4d3c-9c57-8a3c2b1a51c8 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:08.080755 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b379ed3-ef13-4d3c-9c57-8a3c2b1a51c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:08.099640 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-4b379ed3-ef13-4d3c-9c57-8a3c2b1a51c8 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:05:08.101732 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-4b379ed3-ef13-4d3c-9c57-8a3c2b1a51c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:05:08.175989 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 468/962] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:05:08 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 01:05:08.453006 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b5261b-a473-4a49-997c-35f9ad6b5f6c None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:05:08.559908 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:05:08.568594 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b5261b-a473-4a49-997c-35f9ad6b5f6c None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:05:08.607236 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9c31eb-f356-4c67-8151-01029cce020c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:05:08.608555 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9c31eb-f356-4c67-8151-01029cce020c None admin] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:08.609099 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9c31eb-f356-4c67-8151-01029cce020c None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:08.609576 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9c31eb-f356-4c67-8151-01029cce020c None admin] PATH_INFO: `/v3/users` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:08.610425 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9c31eb-f356-4c67-8151-01029cce020c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:05:08.612075 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9c31eb-f356-4c67-8151-01029cce020c None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:05:08.614832 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: WARNING keystone.server.flask.common [None req-5c9c31eb-f356-4c67-8151-01029cce020c None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 31 01:05:08.684931 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.sql.core [None req-5c9c31eb-f356-4c67-8151-01029cce020c None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 31 01:05:08.684931 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 31 01:05:08.684931 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [parameters: {'user_id': '2f92b5dc2387439a80d96f6f729d3543', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 31 01:05:08.684931 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81750) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 31 01:05:08.702538 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application [None req-5c9c31eb-f356-4c67-8151-01029cce020c None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 31 01:05:08.702538 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:05:08.702538 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 31 01:05:08.702538 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 31 01:05:08.702538 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 31 01:05:08.702538 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 31 01:05:08.702538 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 31 01:05:08.702538 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application result = self._query(query) Aug 31 01:05:08.702538 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 31 01:05:08.702538 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application conn.query(q) Aug 31 01:05:08.702538 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 31 01:05:08.702538 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 31 01:05:08.702538 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 31 01:05:08.702538 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application result.read() Aug 31 01:05:08.702538 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 31 01:05:08.703552 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 31 01:05:08.703552 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 31 01:05:08.703552 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 31 01:05:08.703552 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 31 01:05:08.703552 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 31 01:05:08.703552 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 31 01:05:08.703552 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 31 01:05:08.703552 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 31 01:05:08.703552 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Aug 31 01:05:08.703552 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 31 01:05:08.703552 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Aug 31 01:05:08.703552 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:05:08.703552 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 31 01:05:08.703552 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 31 01:05:08.703552 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Aug 31 01:05:08.703552 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Aug 31 01:05:08.703552 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 31 01:05:08.703552 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application next(self.gen) Aug 31 01:05:08.704530 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 31 01:05:08.704530 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application yield resource Aug 31 01:05:08.704530 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 31 01:05:08.704530 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application next(self.gen) Aug 31 01:05:08.704530 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 31 01:05:08.704530 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application self.session.rollback() Aug 31 01:05:08.704530 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 31 01:05:08.704530 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application self.force_reraise() Aug 31 01:05:08.704530 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 31 01:05:08.704530 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application raise self.value Aug 31 01:05:08.704530 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 31 01:05:08.704530 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 31 01:05:08.704530 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 31 01:05:08.704530 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application session.commit() Aug 31 01:05:08.704530 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 31 01:05:08.704530 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 31 01:05:08.704530 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 31 01:05:08.705458 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application self._prepare_impl() Aug 31 01:05:08.705458 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 31 01:05:08.705458 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application self.session.flush() Aug 31 01:05:08.705458 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 31 01:05:08.705458 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application self._flush(objects) Aug 31 01:05:08.705458 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 31 01:05:08.705458 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 31 01:05:08.705458 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 31 01:05:08.705458 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application compat.raise_( Aug 31 01:05:08.705458 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 31 01:05:08.705458 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application raise exception Aug 31 01:05:08.705458 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 31 01:05:08.705458 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application flush_context.execute() Aug 31 01:05:08.705458 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 31 01:05:08.705458 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application rec.execute(self) Aug 31 01:05:08.705458 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 31 01:05:08.705458 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 31 01:05:08.706354 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 31 01:05:08.706354 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 31 01:05:08.706354 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Aug 31 01:05:08.706354 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application result = connection._execute_20( Aug 31 01:05:08.706354 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 31 01:05:08.706354 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 31 01:05:08.706354 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 31 01:05:08.706354 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 31 01:05:08.706354 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 31 01:05:08.706354 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 31 01:05:08.706354 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 31 01:05:08.706354 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 31 01:05:08.706354 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 31 01:05:08.706354 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 31 01:05:08.706354 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 31 01:05:08.706354 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application raise exception Aug 31 01:05:08.707216 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 31 01:05:08.707216 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 31 01:05:08.707216 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 31 01:05:08.707216 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 31 01:05:08.707216 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 31 01:05:08.707216 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application result = self._query(query) Aug 31 01:05:08.707216 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 31 01:05:08.707216 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application conn.query(q) Aug 31 01:05:08.707216 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 31 01:05:08.707216 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 31 01:05:08.707216 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 31 01:05:08.707216 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application result.read() Aug 31 01:05:08.707216 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 31 01:05:08.707216 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 31 01:05:08.707216 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 31 01:05:08.707216 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 31 01:05:08.708045 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 31 01:05:08.708045 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 31 01:05:08.708045 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 31 01:05:08.708045 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 31 01:05:08.708045 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 31 01:05:08.708045 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 31 01:05:08.708045 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application [parameters: {'user_id': '2f92b5dc2387439a80d96f6f729d3543', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 31 01:05:08.708045 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 31 01:05:08.708045 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Aug 31 01:05:08.708045 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 01:05:08.708045 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Aug 31 01:05:08.708045 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:05:08.708045 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:05:08.708045 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:05:08.708045 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:05:08.710706 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:05:08.710706 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:05:08.710706 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:05:08.710706 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:05:08.710706 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:05:08.710706 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:05:08.710706 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:05:08.710706 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Aug 31 01:05:08.710706 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Aug 31 01:05:08.710706 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:05:08.710706 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:05:08.710706 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 01:05:08.710706 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 01:05:08.710706 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 01:05:08.710706 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 01:05:08.710706 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Aug 31 01:05:08.710706 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Aug 31 01:05:08.712874 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Aug 31 01:05:08.712874 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Aug 31 01:05:08.712874 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 31 01:05:08.712874 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 31 01:05:08.712874 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 31 01:05:08.712874 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Aug 31 01:05:08.715443 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 495/963] 10.222.0.20 () {60 vars in 1131 bytes} [Wed Aug 31 01:05:08 2022] POST /identity/v3/users => generated 160 bytes in 263 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:05:08.721146 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b379ed3-ef13-4d3c-9c57-8a3c2b1a51c8 None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:05:08.793386 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:05:08.801634 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84684e50-1c97-475c-bf34-bedabfe26270 None None] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:08.801915 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84684e50-1c97-475c-bf34-bedabfe26270 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:08.802179 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84684e50-1c97-475c-bf34-bedabfe26270 None None] PATH_INFO: `/` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:08.808826 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 496/964] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:05:08 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:05:08.811906 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1864e4-6b0b-4681-b374-c40003647b90 None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:08.812187 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1864e4-6b0b-4681-b374-c40003647b90 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:08.812461 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1864e4-6b0b-4681-b374-c40003647b90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:08.825430 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b379ed3-ef13-4d3c-9c57-8a3c2b1a51c8 None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:05:08.833261 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-1b1864e4-6b0b-4681-b374-c40003647b90 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:05:08.835781 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-1b1864e4-6b0b-4681-b374-c40003647b90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:05:08.862674 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a1a2db-7788-4d1f-8b75-107e4617e194 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:05:08.863426 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a1a2db-7788-4d1f-8b75-107e4617e194 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:08.863715 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a1a2db-7788-4d1f-8b75-107e4617e194 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:08.863996 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a1a2db-7788-4d1f-8b75-107e4617e194 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:08.872121 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a1a2db-7788-4d1f-8b75-107e4617e194 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:05:08.873622 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a1a2db-7788-4d1f-8b75-107e4617e194 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:05:08.887272 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application [None req-39a1a2db-7788-4d1f-8b75-107e4617e194 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 31 01:05:08.887272 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:05:08.887272 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:05:08.887272 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:05:08.887272 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:05:08.887272 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:05:08.887272 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:05:08.887272 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:05:08.887272 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:05:08.887272 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:05:08.887272 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:05:08.887272 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:05:08.887272 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 01:05:08.887272 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 01:05:08.887272 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 01:05:08.887272 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 01:05:08.889599 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:05:08.889599 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:05:08.889599 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 01:05:08.889599 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 01:05:08.889599 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 01:05:08.889599 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 01:05:08.889599 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:05:08.889599 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:05:08.889599 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:05:08.889599 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:05:08.889599 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:05:08.889599 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application with Lock( Aug 31 01:05:08.889599 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:05:08.889599 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:05:08.889599 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:05:08.889599 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:05:08.889599 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:05:08.889599 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:05:08.891939 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:05:08.891939 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:05:08.891939 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 01:05:08.891939 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 01:05:08.891939 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 01:05:08.891939 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 01:05:08.891939 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 01:05:08.891939 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 01:05:08.891939 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 31 01:05:08.891939 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Aug 31 01:05:08.891939 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 469/965] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Aug 31 01:05:08 2022] GET /identity/v3/users/cinder => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:05:08.900831 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a1a2db-7788-4d1f-8b75-107e4617e194 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:05:08.914382 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 497/966] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:05:08 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 01:05:09.006967 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a1a2db-7788-4d1f-8b75-107e4617e194 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:05:09.036050 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:05:09.042508 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f48763a-f498-466b-8078-66633fffd84a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:05:09.043343 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f48763a-f498-466b-8078-66633fffd84a None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:09.043844 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f48763a-f498-466b-8078-66633fffd84a None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:09.044301 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f48763a-f498-466b-8078-66633fffd84a None admin] PATH_INFO: `/v3/users` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:09.045278 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f48763a-f498-466b-8078-66633fffd84a None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:05:09.045770 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f48763a-f498-466b-8078-66633fffd84a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:05:09.047388 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f48763a-f498-466b-8078-66633fffd84a None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:05:09.085663 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 470/967] 10.222.0.20 () {56 vars in 1101 bytes} [Wed Aug 31 01:05:08 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 01:05:09.183315 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1864e4-6b0b-4681-b374-c40003647b90 None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:05:09.195756 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdee467-0554-415a-b47b-4b2ce940b7ce None None] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:09.196045 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdee467-0554-415a-b47b-4b2ce940b7ce None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:09.196674 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdee467-0554-415a-b47b-4b2ce940b7ce None None] PATH_INFO: `/` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:09.198730 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 471/968] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:05:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 01:05:09.204408 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23f7838-80aa-4e18-a0dd-00702657db28 None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:09.204693 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23f7838-80aa-4e18-a0dd-00702657db28 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:09.205003 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23f7838-80aa-4e18-a0dd-00702657db28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:09.225747 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-c23f7838-80aa-4e18-a0dd-00702657db28 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:05:09.228207 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-c23f7838-80aa-4e18-a0dd-00702657db28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:05:09.287574 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:05:09.303285 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1864e4-6b0b-4681-b374-c40003647b90 None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:05:09.311670 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 472/969] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:05:09 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 01:05:09.313158 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cc693b-5cbe-4f38-b6d5-51ee1764ea6e None None] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:09.313455 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cc693b-5cbe-4f38-b6d5-51ee1764ea6e None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:09.314035 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cc693b-5cbe-4f38-b6d5-51ee1764ea6e None None] PATH_INFO: `/` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:09.315082 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 473/970] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:05:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:05:09.320645 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7735694-a5d4-4be8-b202-1c4b175978a1 None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:09.320993 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7735694-a5d4-4be8-b202-1c4b175978a1 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:09.321278 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7735694-a5d4-4be8-b202-1c4b175978a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:09.344271 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-c7735694-a5d4-4be8-b202-1c4b175978a1 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:05:09.344895 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b50fd3-9ffc-47d9-86f4-977b07fe261a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:05:09.345510 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-c7735694-a5d4-4be8-b202-1c4b175978a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:05:09.346083 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b50fd3-9ffc-47d9-86f4-977b07fe261a None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:09.346826 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b50fd3-9ffc-47d9-86f4-977b07fe261a None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:09.347112 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b50fd3-9ffc-47d9-86f4-977b07fe261a None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:09.366802 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b50fd3-9ffc-47d9-86f4-977b07fe261a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 01:05:09.373024 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application [None req-88b50fd3-9ffc-47d9-86f4-977b07fe261a None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 31 01:05:09.373024 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:05:09.373024 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:05:09.373024 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:05:09.373024 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:05:09.373024 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:05:09.373024 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:05:09.373024 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:05:09.373024 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:05:09.373024 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:05:09.373024 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:05:09.373024 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:05:09.373024 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 01:05:09.373024 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 01:05:09.373024 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 01:05:09.373024 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 01:05:09.374154 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 01:05:09.374154 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 01:05:09.374154 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 01:05:09.374154 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 01:05:09.374154 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:05:09.374154 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:05:09.374154 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:05:09.374154 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:05:09.374154 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:05:09.374154 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:05:09.374154 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:05:09.374154 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application with Lock( Aug 31 01:05:09.374154 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:05:09.374154 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:05:09.374154 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:05:09.374154 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:05:09.375178 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:05:09.375178 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:05:09.375178 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:05:09.375178 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:05:09.375178 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 01:05:09.375178 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 01:05:09.375178 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 01:05:09.375178 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 01:05:09.375178 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 01:05:09.375178 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 01:05:09.375178 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 31 01:05:09.375178 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Aug 31 01:05:09.377729 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 498/971] 10.222.0.20 () {56 vars in 1120 bytes} [Wed Aug 31 01:05:09 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:05:09.383715 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b50fd3-9ffc-47d9-86f4-977b07fe261a None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:05:09.426441 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 474/972] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:05:09 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 01:05:09.486417 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b50fd3-9ffc-47d9-86f4-977b07fe261a None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:05:09.519038 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169a28a9-ee52-4ad3-a1bd-9ff7aa37d3f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:05:09.519639 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169a28a9-ee52-4ad3-a1bd-9ff7aa37d3f5 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:09.519917 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169a28a9-ee52-4ad3-a1bd-9ff7aa37d3f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:09.520188 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169a28a9-ee52-4ad3-a1bd-9ff7aa37d3f5 None admin] PATH_INFO: `/v3/roles` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:09.520860 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169a28a9-ee52-4ad3-a1bd-9ff7aa37d3f5 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:05:09.521161 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169a28a9-ee52-4ad3-a1bd-9ff7aa37d3f5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:05:09.522684 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169a28a9-ee52-4ad3-a1bd-9ff7aa37d3f5 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:05:09.535327 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 499/973] 10.222.0.20 () {56 vars in 1115 bytes} [Wed Aug 31 01:05:09 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:05:09.536749 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7735694-a5d4-4be8-b202-1c4b175978a1 None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:05:09.585497 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:05:09.588286 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169a28a9-ee52-4ad3-a1bd-9ff7aa37d3f5 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:05:09.599202 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:05:09.640471 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7735694-a5d4-4be8-b202-1c4b175978a1 None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:05:09.674228 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6bbc33-4413-4134-9f04-b504782eb5e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:05:09.674854 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6bbc33-4413-4134-9f04-b504782eb5e1 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:09.675132 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6bbc33-4413-4134-9f04-b504782eb5e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:09.675399 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6bbc33-4413-4134-9f04-b504782eb5e1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:09.685948 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6bbc33-4413-4134-9f04-b504782eb5e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:05:09.687391 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6bbc33-4413-4134-9f04-b504782eb5e1 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:05:09.693715 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169a28a9-ee52-4ad3-a1bd-9ff7aa37d3f5 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:05:09.702481 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application [None req-1e6bbc33-4413-4134-9f04-b504782eb5e1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 01:05:09.702481 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:05:09.702481 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:05:09.702481 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:05:09.702481 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:05:09.702481 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:05:09.702481 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:05:09.702481 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:05:09.702481 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:05:09.702481 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:05:09.702481 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:05:09.702481 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:05:09.702481 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 01:05:09.702481 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 01:05:09.702481 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 01:05:09.702481 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 01:05:09.703492 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:05:09.703492 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:05:09.703492 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:05:09.703492 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:05:09.703492 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:05:09.703492 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:05:09.703492 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:05:09.703492 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application with Lock( Aug 31 01:05:09.703492 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:05:09.703492 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:05:09.703492 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:05:09.703492 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:05:09.703492 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:05:09.703492 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:05:09.703492 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:05:09.703492 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:05:09.703492 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 01:05:09.704399 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 01:05:09.704399 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 01:05:09.704399 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Aug 31 01:05:09.704399 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 475/974] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 01:05:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 01:05:09.704644 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6bbc33-4413-4134-9f04-b504782eb5e1 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:05:09.730718 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e32e8b-8a64-4dfe-ba87-2f8ba2380611 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:05:09.731429 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e32e8b-8a64-4dfe-ba87-2f8ba2380611 None admin] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:09.731712 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e32e8b-8a64-4dfe-ba87-2f8ba2380611 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:09.731987 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e32e8b-8a64-4dfe-ba87-2f8ba2380611 None admin] PATH_INFO: `/v3/services` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:09.732635 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e32e8b-8a64-4dfe-ba87-2f8ba2380611 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:05:09.734022 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e32e8b-8a64-4dfe-ba87-2f8ba2380611 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:05:09.799633 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:05:09.802492 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6bbc33-4413-4134-9f04-b504782eb5e1 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:05:09.822853 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 500/975] 10.222.0.20 () {60 vars in 1141 bytes} [Wed Aug 31 01:05:09 2022] POST /identity/v3/services => generated 242 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 01:05:09.830253 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e32e8b-8a64-4dfe-ba87-2f8ba2380611 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:05:09.846584 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02317c3-06dc-41dd-8ddc-be067422cc13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:05:09.847157 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02317c3-06dc-41dd-8ddc-be067422cc13 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:09.847711 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02317c3-06dc-41dd-8ddc-be067422cc13 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:09.847995 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02317c3-06dc-41dd-8ddc-be067422cc13 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:09.848592 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02317c3-06dc-41dd-8ddc-be067422cc13 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:05:09.850768 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02317c3-06dc-41dd-8ddc-be067422cc13 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:05:09.861649 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application [None req-e02317c3-06dc-41dd-8ddc-be067422cc13 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 01:05:09.861649 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:05:09.861649 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 01:05:09.861649 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 01:05:09.861649 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 01:05:09.861649 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 01:05:09.861649 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 01:05:09.861649 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 01:05:09.861649 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 01:05:09.861649 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Aug 31 01:05:09.861649 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 01:05:09.861649 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Aug 31 01:05:09.861649 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:05:09.861649 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:05:09.861649 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:05:09.861649 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:05:09.861649 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:05:09.862744 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:05:09.862744 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:05:09.862744 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:05:09.862744 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:05:09.862744 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:05:09.862744 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:05:09.862744 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 01:05:09.862744 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 01:05:09.862744 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 01:05:09.862744 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 01:05:09.862744 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:05:09.862744 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:05:09.862744 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:05:09.862744 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:05:09.862744 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:05:09.862744 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:05:09.863618 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:05:09.863618 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application with Lock( Aug 31 01:05:09.863618 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:05:09.863618 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:05:09.863618 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:05:09.863618 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:05:09.863618 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:05:09.863618 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:05:09.863618 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:05:09.863618 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:05:09.863618 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 01:05:09.863618 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 01:05:09.863618 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 01:05:09.863618 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Aug 31 01:05:09.868432 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 476/976] 10.222.0.20 () {56 vars in 1126 bytes} [Wed Aug 31 01:05:09 2022] GET /identity/v3/services/object-store => generated 93 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:05:09.869486 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02317c3-06dc-41dd-8ddc-be067422cc13 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:05:09.916174 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:05:09.953926 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e32e8b-8a64-4dfe-ba87-2f8ba2380611 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:05:09.990296 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02317c3-06dc-41dd-8ddc-be067422cc13 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:05:09.993797 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1596a000-be85-4a23-9833-77ea787db9bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:05:09.993797 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1596a000-be85-4a23-9833-77ea787db9bf None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:09.993797 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1596a000-be85-4a23-9833-77ea787db9bf None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:09.994078 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1596a000-be85-4a23-9833-77ea787db9bf None admin] PATH_INFO: `/v3/domains` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:09.995066 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1596a000-be85-4a23-9833-77ea787db9bf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:05:09.995066 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1596a000-be85-4a23-9833-77ea787db9bf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:05:09.996318 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1596a000-be85-4a23-9833-77ea787db9bf None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:05:10.009006 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 501/977] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 31 01:05:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:05:10.015218 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1596a000-be85-4a23-9833-77ea787db9bf None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:05:10.025991 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2427a899-c594-44e7-ba32-aa6f3d417ca4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:05:10.026598 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2427a899-c594-44e7-ba32-aa6f3d417ca4 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:10.027081 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2427a899-c594-44e7-ba32-aa6f3d417ca4 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:10.027190 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2427a899-c594-44e7-ba32-aa6f3d417ca4 None admin] PATH_INFO: `/v3/services` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:10.027766 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2427a899-c594-44e7-ba32-aa6f3d417ca4 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:05:10.028070 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2427a899-c594-44e7-ba32-aa6f3d417ca4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:05:10.029512 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2427a899-c594-44e7-ba32-aa6f3d417ca4 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:05:10.034316 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 477/978] 10.222.0.20 () {56 vars in 1122 bytes} [Wed Aug 31 01:05:09 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:05:10.040749 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2427a899-c594-44e7-ba32-aa6f3d417ca4 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:05:10.069550 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:05:10.121211 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1596a000-be85-4a23-9833-77ea787db9bf None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:05:10.149766 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:05:10.154458 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2427a899-c594-44e7-ba32-aa6f3d417ca4 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:05:10.157647 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8527c353-9f3e-42e9-8f9f-ce29957cef8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:05:10.158282 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8527c353-9f3e-42e9-8f9f-ce29957cef8e None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:10.164011 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8527c353-9f3e-42e9-8f9f-ce29957cef8e None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:10.164011 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8527c353-9f3e-42e9-8f9f-ce29957cef8e None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:10.166247 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8527c353-9f3e-42e9-8f9f-ce29957cef8e None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:05:10.167695 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8527c353-9f3e-42e9-8f9f-ce29957cef8e None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:05:10.180339 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application [None req-8527c353-9f3e-42e9-8f9f-ce29957cef8e None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 31 01:05:10.180339 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:05:10.180339 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:05:10.180339 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:05:10.180339 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:05:10.180339 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:05:10.180339 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:05:10.180339 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:05:10.180339 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:05:10.180339 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:05:10.180339 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:05:10.180339 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:05:10.180339 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 01:05:10.180339 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 01:05:10.180339 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 01:05:10.180339 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 01:05:10.181375 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:05:10.181375 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:05:10.181375 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 01:05:10.181375 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 01:05:10.181375 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 01:05:10.181375 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 01:05:10.181375 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:05:10.181375 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:05:10.181375 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:05:10.181375 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:05:10.181375 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:05:10.181375 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application with Lock( Aug 31 01:05:10.181375 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:05:10.181375 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:05:10.181375 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:05:10.181375 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:05:10.181375 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:05:10.181375 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:05:10.182926 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:05:10.182926 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:05:10.182926 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 01:05:10.182926 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 01:05:10.182926 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 01:05:10.182926 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 01:05:10.182926 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 01:05:10.182926 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 01:05:10.182926 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 31 01:05:10.182926 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Aug 31 01:05:10.182926 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 502/979] 10.222.0.20 () {56 vars in 1117 bytes} [Wed Aug 31 01:05:10 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 01:05:10.187264 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8527c353-9f3e-42e9-8f9f-ce29957cef8e None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:05:10.195972 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed561f35-ad09-4a99-864d-77f348f9b729 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:05:10.196891 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed561f35-ad09-4a99-864d-77f348f9b729 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:10.196891 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed561f35-ad09-4a99-864d-77f348f9b729 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:10.197188 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed561f35-ad09-4a99-864d-77f348f9b729 None admin] PATH_INFO: `/v3/services` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:10.197806 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed561f35-ad09-4a99-864d-77f348f9b729 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:05:10.198115 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed561f35-ad09-4a99-864d-77f348f9b729 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:05:10.199481 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed561f35-ad09-4a99-864d-77f348f9b729 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:05:10.204583 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 478/980] 10.222.0.20 () {56 vars in 1122 bytes} [Wed Aug 31 01:05:10 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 01:05:10.210956 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed561f35-ad09-4a99-864d-77f348f9b729 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:05:10.293290 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8527c353-9f3e-42e9-8f9f-ce29957cef8e None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:05:10.302487 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:05:10.316021 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed561f35-ad09-4a99-864d-77f348f9b729 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:05:10.328830 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30fc1d3-2b26-4105-b441-ef71c59df10f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:05:10.329447 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30fc1d3-2b26-4105-b441-ef71c59df10f None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:10.329727 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30fc1d3-2b26-4105-b441-ef71c59df10f None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:10.330002 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30fc1d3-2b26-4105-b441-ef71c59df10f None admin] PATH_INFO: `/v3/users` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:10.330826 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30fc1d3-2b26-4105-b441-ef71c59df10f None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:05:10.331134 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30fc1d3-2b26-4105-b441-ef71c59df10f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:05:10.332560 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30fc1d3-2b26-4105-b441-ef71c59df10f None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:05:10.350668 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 503/981] 10.222.0.20 () {56 vars in 1149 bytes} [Wed Aug 31 01:05:10 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 01:05:10.352922 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d120820-0e69-4cc1-b6e5-b54d52cd16f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:05:10.353694 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d120820-0e69-4cc1-b6e5-b54d52cd16f7 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:10.354118 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d120820-0e69-4cc1-b6e5-b54d52cd16f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:10.354563 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d120820-0e69-4cc1-b6e5-b54d52cd16f7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:10.355486 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d120820-0e69-4cc1-b6e5-b54d52cd16f7 None admin] RBAC: Adding query filter params (interface=admin, service_id=81a6706394f14a96aecda72ded4f638c, region_id=RegionOne) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:05:10.355947 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d120820-0e69-4cc1-b6e5-b54d52cd16f7 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:05:10.356157 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30fc1d3-2b26-4105-b441-ef71c59df10f None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:05:10.357738 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d120820-0e69-4cc1-b6e5-b54d52cd16f7 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:05:10.363504 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 479/982] 10.222.0.20 () {56 vars in 1249 bytes} [Wed Aug 31 01:05:10 2022] GET /identity/v3/endpoints?service_id=81a6706394f14a96aecda72ded4f638c&interface=admin®ion_id=RegionOne => generated 193 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 01:05:10.369171 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d120820-0e69-4cc1-b6e5-b54d52cd16f7 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:05:10.411149 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:05:10.466939 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30fc1d3-2b26-4105-b441-ef71c59df10f None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:05:10.478553 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d120820-0e69-4cc1-b6e5-b54d52cd16f7 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:05:10.501155 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6106d5d5-00bf-42b0-bf48-007a072a8372 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:05:10.501872 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6106d5d5-00bf-42b0-bf48-007a072a8372 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:10.502149 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6106d5d5-00bf-42b0-bf48-007a072a8372 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:10.502413 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6106d5d5-00bf-42b0-bf48-007a072a8372 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:10.516008 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a649ab5-fa85-437b-a3ba-e03a5c813bec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:05:10.516693 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a649ab5-fa85-437b-a3ba-e03a5c813bec None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:10.517013 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a649ab5-fa85-437b-a3ba-e03a5c813bec None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:10.517293 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a649ab5-fa85-437b-a3ba-e03a5c813bec None admin] PATH_INFO: `/v3/services` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:10.517899 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a649ab5-fa85-437b-a3ba-e03a5c813bec None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:05:10.519285 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a649ab5-fa85-437b-a3ba-e03a5c813bec None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:05:10.525787 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 480/983] 10.222.0.20 () {56 vars in 1087 bytes} [Wed Aug 31 01:05:10 2022] GET /identity/v3/services => generated 1727 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 01:05:10.577762 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6106d5d5-00bf-42b0-bf48-007a072a8372 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:05:10.579243 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6106d5d5-00bf-42b0-bf48-007a072a8372 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:05:10.627277 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:05:10.687084 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 504/984] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Aug 31 01:05:10 2022] GET /identity/v3/auth/tokens => generated 1570 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 31 01:05:10.693385 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a649ab5-fa85-437b-a3ba-e03a5c813bec None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:05:10.712606 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:05:10.793783 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a649ab5-fa85-437b-a3ba-e03a5c813bec None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:05:10.827998 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988d188f-faa7-4d66-9988-2e995d080e2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:05:10.828634 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988d188f-faa7-4d66-9988-2e995d080e2b None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:10.828939 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988d188f-faa7-4d66-9988-2e995d080e2b None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:10.829214 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988d188f-faa7-4d66-9988-2e995d080e2b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:10.839987 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988d188f-faa7-4d66-9988-2e995d080e2b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:05:10.841458 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988d188f-faa7-4d66-9988-2e995d080e2b None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:05:10.856407 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application [None req-988d188f-faa7-4d66-9988-2e995d080e2b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 01:05:10.856407 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:05:10.856407 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:05:10.856407 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:05:10.856407 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:05:10.856407 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:05:10.856407 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:05:10.856407 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:05:10.856407 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:05:10.856407 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:05:10.856407 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:05:10.856407 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:05:10.856407 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 01:05:10.856407 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 01:05:10.856407 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 01:05:10.856407 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 01:05:10.857578 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:05:10.857578 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:05:10.857578 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:05:10.857578 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:05:10.857578 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:05:10.857578 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:05:10.857578 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:05:10.857578 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application with Lock( Aug 31 01:05:10.857578 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:05:10.857578 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:05:10.857578 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:05:10.857578 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:05:10.857578 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:05:10.857578 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:05:10.857578 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:05:10.857578 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:05:10.857578 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 01:05:10.858493 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 01:05:10.858493 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 01:05:10.858493 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Aug 31 01:05:10.858493 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 481/985] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 01:05:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:05:10.863110 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6106d5d5-00bf-42b0-bf48-007a072a8372 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:05:10.974139 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6106d5d5-00bf-42b0-bf48-007a072a8372 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:05:11.010883 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13297f8e-0d05-4e5e-b3a4-197c18b61be4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:05:11.011490 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13297f8e-0d05-4e5e-b3a4-197c18b61be4 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:11.011771 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13297f8e-0d05-4e5e-b3a4-197c18b61be4 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:11.012044 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13297f8e-0d05-4e5e-b3a4-197c18b61be4 None admin] PATH_INFO: `/v3/domains` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:11.012678 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13297f8e-0d05-4e5e-b3a4-197c18b61be4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:05:11.013011 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13297f8e-0d05-4e5e-b3a4-197c18b61be4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:05:11.014400 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13297f8e-0d05-4e5e-b3a4-197c18b61be4 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:05:11.025871 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 505/986] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 31 01:05:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:05:11.031163 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988d188f-faa7-4d66-9988-2e995d080e2b None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:05:11.097827 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:05:11.132819 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988d188f-faa7-4d66-9988-2e995d080e2b None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:05:11.168127 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b593a221-5c1d-412f-930f-8fa98635a789 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:05:11.168886 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b593a221-5c1d-412f-930f-8fa98635a789 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:11.169066 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b593a221-5c1d-412f-930f-8fa98635a789 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:11.169321 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b593a221-5c1d-412f-930f-8fa98635a789 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:11.176054 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b593a221-5c1d-412f-930f-8fa98635a789 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:05:11.177601 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b593a221-5c1d-412f-930f-8fa98635a789 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:05:11.190165 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application [None req-b593a221-5c1d-412f-930f-8fa98635a789 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 01:05:11.190165 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:05:11.190165 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:05:11.190165 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:05:11.190165 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:05:11.190165 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:05:11.190165 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:05:11.190165 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:05:11.190165 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:05:11.190165 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:05:11.190165 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:05:11.190165 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:05:11.190165 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 01:05:11.190165 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 01:05:11.190165 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 01:05:11.191051 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 01:05:11.191051 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:05:11.191051 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:05:11.191051 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:05:11.191051 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:05:11.191051 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:05:11.191051 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:05:11.191051 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:05:11.191051 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application with Lock( Aug 31 01:05:11.191051 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:05:11.191051 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:05:11.191051 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:05:11.191051 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:05:11.191051 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:05:11.191051 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:05:11.191051 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:05:11.191051 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:05:11.191950 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 01:05:11.191950 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 01:05:11.191950 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 01:05:11.191950 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 01:05:11.191950 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 01:05:11.191950 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 01:05:11.191950 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 01:05:11.191950 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Aug 31 01:05:11.191950 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 482/987] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Aug 31 01:05:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:05:11.198592 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13297f8e-0d05-4e5e-b3a4-197c18b61be4 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:05:11.208626 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:05:11.301814 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13297f8e-0d05-4e5e-b3a4-197c18b61be4 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:05:11.335045 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da5d1e4-78f7-46e0-996f-338c9f6021c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:05:11.335652 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da5d1e4-78f7-46e0-996f-338c9f6021c2 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:11.335932 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da5d1e4-78f7-46e0-996f-338c9f6021c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:11.336206 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da5d1e4-78f7-46e0-996f-338c9f6021c2 None admin] PATH_INFO: `/v3/projects` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:11.336948 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da5d1e4-78f7-46e0-996f-338c9f6021c2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:05:11.337257 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da5d1e4-78f7-46e0-996f-338c9f6021c2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:05:11.338685 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da5d1e4-78f7-46e0-996f-338c9f6021c2 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:05:11.351219 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 506/988] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Aug 31 01:05:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:05:11.356991 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b593a221-5c1d-412f-930f-8fa98635a789 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:05:11.463283 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b593a221-5c1d-412f-930f-8fa98635a789 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:05:11.491047 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:05:11.498885 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9d6af2-1a79-48d3-8205-4a53f2c8818b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:05:11.499502 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9d6af2-1a79-48d3-8205-4a53f2c8818b None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:11.499787 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9d6af2-1a79-48d3-8205-4a53f2c8818b None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:11.500123 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9d6af2-1a79-48d3-8205-4a53f2c8818b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:11.509494 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9d6af2-1a79-48d3-8205-4a53f2c8818b None admin] RBAC: Adding query filter params (role.id=586b679002e645439ed44fe1ca4a7cc6, scope.project.id=60b2abc15c2849ba8a1ed92117cc981b, user.id=b3efb4f9daac4a0ca05a36dd361f17e5) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:05:11.509494 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9d6af2-1a79-48d3-8205-4a53f2c8818b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:05:11.509494 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9d6af2-1a79-48d3-8205-4a53f2c8818b None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:05:11.509494 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 483/989] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Aug 31 01:05:11 2022] GET /identity/v3/role_assignments?user.id=b3efb4f9daac4a0ca05a36dd361f17e5&scope.project.id=60b2abc15c2849ba8a1ed92117cc981b&role.id=586b679002e645439ed44fe1ca4a7cc6 => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:05:11.782355 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8aa0434-2a49-4a37-bd5f-a6133c0e84b9 None None] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:11.782778 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8aa0434-2a49-4a37-bd5f-a6133c0e84b9 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:11.782979 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8aa0434-2a49-4a37-bd5f-a6133c0e84b9 None None] PATH_INFO: `/` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:11.784394 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 507/990] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:05:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:05:11.789441 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488aa6cd-69df-4a9b-851e-8971ffa36570 None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:11.789737 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488aa6cd-69df-4a9b-851e-8971ffa36570 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:11.790025 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488aa6cd-69df-4a9b-851e-8971ffa36570 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:11.808929 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-488aa6cd-69df-4a9b-851e-8971ffa36570 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:05:11.810340 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-488aa6cd-69df-4a9b-851e-8971ffa36570 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:05:11.879263 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 484/991] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:05:11 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 01:05:12.152086 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8aa0434-2a49-4a37-bd5f-a6133c0e84b9 None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:05:12.219490 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:05:12.255024 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8aa0434-2a49-4a37-bd5f-a6133c0e84b9 None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:05:12.287966 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5dbe44-776a-48fd-beb4-8f2f3e97e44e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:05:12.288587 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5dbe44-776a-48fd-beb4-8f2f3e97e44e None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:12.288912 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5dbe44-776a-48fd-beb4-8f2f3e97e44e None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:12.289182 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5dbe44-776a-48fd-beb4-8f2f3e97e44e None admin] PATH_INFO: `/v3/services/compute` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:12.289751 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5dbe44-776a-48fd-beb4-8f2f3e97e44e None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:05:12.291089 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5dbe44-776a-48fd-beb4-8f2f3e97e44e None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:05:12.299717 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application [None req-ce5dbe44-776a-48fd-beb4-8f2f3e97e44e None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 31 01:05:12.299717 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:05:12.299717 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 01:05:12.299717 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 01:05:12.299717 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 01:05:12.299717 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 01:05:12.299717 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 01:05:12.299717 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 01:05:12.299717 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 31 01:05:12.299717 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Aug 31 01:05:12.299717 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 01:05:12.299717 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Aug 31 01:05:12.299717 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:05:12.299717 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:05:12.299717 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:05:12.299717 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:05:12.299717 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:05:12.300708 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:05:12.300708 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:05:12.300708 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:05:12.300708 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:05:12.300708 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:05:12.300708 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:05:12.300708 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 01:05:12.300708 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 01:05:12.300708 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 01:05:12.300708 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 01:05:12.300708 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:05:12.300708 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:05:12.300708 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:05:12.300708 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:05:12.300708 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:05:12.300708 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:05:12.302252 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:05:12.302252 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application with Lock( Aug 31 01:05:12.302252 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:05:12.302252 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:05:12.302252 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:05:12.302252 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:05:12.302252 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:05:12.302252 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:05:12.302252 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:05:12.302252 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:05:12.302252 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 01:05:12.302252 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 01:05:12.302252 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 31 01:05:12.302252 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Aug 31 01:05:12.302252 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 508/992] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Aug 31 01:05:12 2022] GET /identity/v3/services/compute => generated 88 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:05:12.307193 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488aa6cd-69df-4a9b-851e-8971ffa36570 None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:05:12.410245 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488aa6cd-69df-4a9b-851e-8971ffa36570 None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:05:12.434298 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:05:12.445493 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247f4595-3d39-4907-95bf-9a4597a050b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:05:12.446108 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247f4595-3d39-4907-95bf-9a4597a050b3 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:12.446391 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247f4595-3d39-4907-95bf-9a4597a050b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:12.446669 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247f4595-3d39-4907-95bf-9a4597a050b3 None admin] PATH_INFO: `/v3/services` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:12.447299 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247f4595-3d39-4907-95bf-9a4597a050b3 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:05:12.447775 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247f4595-3d39-4907-95bf-9a4597a050b3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:05:12.449211 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247f4595-3d39-4907-95bf-9a4597a050b3 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:05:12.454535 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 485/993] 10.222.0.20 () {56 vars in 1112 bytes} [Wed Aug 31 01:05:12 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 01:05:12.460141 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5dbe44-776a-48fd-beb4-8f2f3e97e44e None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:05:12.462484 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c837dbe5-8015-4da7-9c8c-be6a7865e513 None None] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:12.462777 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c837dbe5-8015-4da7-9c8c-be6a7865e513 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:12.463061 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c837dbe5-8015-4da7-9c8c-be6a7865e513 None None] PATH_INFO: `/` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:12.464339 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 486/994] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:05:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 01:05:12.469134 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0719aff-954c-4633-bcfc-c91646cafbb0 None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:12.469422 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0719aff-954c-4633-bcfc-c91646cafbb0 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:12.469700 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0719aff-954c-4633-bcfc-c91646cafbb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:12.488289 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-c0719aff-954c-4633-bcfc-c91646cafbb0 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:05:12.489570 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-c0719aff-954c-4633-bcfc-c91646cafbb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:05:12.560142 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 487/995] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:05:12 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 01:05:12.573428 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5dbe44-776a-48fd-beb4-8f2f3e97e44e None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:05:12.603591 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:05:12.609402 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa868977-4bb1-4261-99f3-e96c780196e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:05:12.610020 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa868977-4bb1-4261-99f3-e96c780196e8 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:12.610465 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa868977-4bb1-4261-99f3-e96c780196e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:12.610654 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa868977-4bb1-4261-99f3-e96c780196e8 None admin] PATH_INFO: `/v3/services` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:12.611291 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa868977-4bb1-4261-99f3-e96c780196e8 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:05:12.611598 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa868977-4bb1-4261-99f3-e96c780196e8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:05:12.613031 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa868977-4bb1-4261-99f3-e96c780196e8 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:05:12.624322 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 509/996] 10.222.0.20 () {56 vars in 1112 bytes} [Wed Aug 31 01:05:12 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:05:12.625652 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0719aff-954c-4633-bcfc-c91646cafbb0 None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:05:12.755599 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0719aff-954c-4633-bcfc-c91646cafbb0 None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:05:12.784085 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:05:12.813481 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472d9228-3ff0-491a-8abf-f93d3f5f99f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:05:12.814519 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472d9228-3ff0-491a-8abf-f93d3f5f99f9 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:12.815018 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472d9228-3ff0-491a-8abf-f93d3f5f99f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:12.815507 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472d9228-3ff0-491a-8abf-f93d3f5f99f9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:12.816865 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472d9228-3ff0-491a-8abf-f93d3f5f99f9 None admin] RBAC: Adding query filter params (interface=public, service_id=6b5b28b2fcd8403a952eceac771e50f4, region_id=RegionOne) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:05:12.817812 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472d9228-3ff0-491a-8abf-f93d3f5f99f9 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:05:12.821110 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472d9228-3ff0-491a-8abf-f93d3f5f99f9 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:05:12.831744 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa868977-4bb1-4261-99f3-e96c780196e8 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:05:12.832046 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 488/997] 10.222.0.20 () {56 vars in 1251 bytes} [Wed Aug 31 01:05:12 2022] GET /identity/v3/endpoints?service_id=6b5b28b2fcd8403a952eceac771e50f4&interface=public®ion_id=RegionOne => generated 194 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:05:12.836200 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472d9228-3ff0-491a-8abf-f93d3f5f99f9 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:05:12.945031 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472d9228-3ff0-491a-8abf-f93d3f5f99f9 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:05:12.981950 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9131aea6-1cea-46cf-b56e-9b9207131246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:05:12.982557 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9131aea6-1cea-46cf-b56e-9b9207131246 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:12.982846 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9131aea6-1cea-46cf-b56e-9b9207131246 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:12.983123 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9131aea6-1cea-46cf-b56e-9b9207131246 None admin] PATH_INFO: `/v3/services` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:12.983727 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9131aea6-1cea-46cf-b56e-9b9207131246 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:05:12.985129 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9131aea6-1cea-46cf-b56e-9b9207131246 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:05:12.991489 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 489/998] 10.222.0.20 () {56 vars in 1087 bytes} [Wed Aug 31 01:05:12 2022] GET /identity/v3/services => generated 1727 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 01:05:13.009151 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa868977-4bb1-4261-99f3-e96c780196e8 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:05:13.072602 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b1b307-61b4-4c4f-9834-34066c90a2e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:05:13.073732 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b1b307-61b4-4c4f-9834-34066c90a2e6 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:13.074232 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b1b307-61b4-4c4f-9834-34066c90a2e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:13.074978 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b1b307-61b4-4c4f-9834-34066c90a2e6 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:13.076408 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b1b307-61b4-4c4f-9834-34066c90a2e6 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:05:13.079037 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b1b307-61b4-4c4f-9834-34066c90a2e6 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:05:13.097044 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application [None req-61b1b307-61b4-4c4f-9834-34066c90a2e6 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 01:05:13.097044 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:05:13.097044 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 01:05:13.097044 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 01:05:13.097044 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 01:05:13.097044 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 01:05:13.097044 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 01:05:13.097044 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 01:05:13.097044 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 01:05:13.097044 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Aug 31 01:05:13.097044 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 01:05:13.097044 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Aug 31 01:05:13.097044 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:05:13.097044 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:05:13.097044 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:05:13.097044 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:05:13.097044 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:05:13.098122 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:05:13.098122 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:05:13.098122 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:05:13.098122 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:05:13.098122 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:05:13.098122 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:05:13.098122 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 01:05:13.098122 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 01:05:13.098122 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 01:05:13.098122 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 01:05:13.098122 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:05:13.098122 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:05:13.098122 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:05:13.098122 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:05:13.098122 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:05:13.098122 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:05:13.099000 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:05:13.099000 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application with Lock( Aug 31 01:05:13.099000 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:05:13.099000 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:05:13.099000 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:05:13.099000 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:05:13.099000 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:05:13.099000 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:05:13.099000 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:05:13.099000 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:05:13.099000 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 01:05:13.099000 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 01:05:13.099000 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 01:05:13.099000 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Aug 31 01:05:13.100396 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 510/999] 10.222.0.20 () {56 vars in 1126 bytes} [Wed Aug 31 01:05:12 2022] GET /identity/v3/services/object-store => generated 93 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:05:13.105811 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9131aea6-1cea-46cf-b56e-9b9207131246 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:05:13.156602 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:05:13.211360 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9131aea6-1cea-46cf-b56e-9b9207131246 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:05:13.244452 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0a52ac-3a3a-4767-a5f5-0e92e35b9318 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:05:13.245096 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0a52ac-3a3a-4767-a5f5-0e92e35b9318 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:13.245376 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0a52ac-3a3a-4767-a5f5-0e92e35b9318 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:13.245648 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0a52ac-3a3a-4767-a5f5-0e92e35b9318 None admin] PATH_INFO: `/v3/services` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:13.246268 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0a52ac-3a3a-4767-a5f5-0e92e35b9318 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:05:13.246568 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0a52ac-3a3a-4767-a5f5-0e92e35b9318 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:05:13.247934 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0a52ac-3a3a-4767-a5f5-0e92e35b9318 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:05:13.252838 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 490/1000] 10.222.0.20 () {56 vars in 1122 bytes} [Wed Aug 31 01:05:13 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:05:13.259305 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b1b307-61b4-4c4f-9834-34066c90a2e6 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:05:13.320437 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:05:13.362870 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b1b307-61b4-4c4f-9834-34066c90a2e6 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:05:13.396056 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3e2e11-3a19-4b1b-a31b-57e6df43f0dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:05:13.397249 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3e2e11-3a19-4b1b-a31b-57e6df43f0dd None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:13.397534 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3e2e11-3a19-4b1b-a31b-57e6df43f0dd None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:13.397811 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3e2e11-3a19-4b1b-a31b-57e6df43f0dd None admin] PATH_INFO: `/v3/services` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:13.398440 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3e2e11-3a19-4b1b-a31b-57e6df43f0dd None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:05:13.398745 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3e2e11-3a19-4b1b-a31b-57e6df43f0dd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:05:13.400112 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3e2e11-3a19-4b1b-a31b-57e6df43f0dd None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:05:13.409640 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 511/1001] 10.222.0.20 () {56 vars in 1122 bytes} [Wed Aug 31 01:05:13 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:05:13.410772 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0a52ac-3a3a-4767-a5f5-0e92e35b9318 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:05:13.513801 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0a52ac-3a3a-4767-a5f5-0e92e35b9318 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:05:13.523107 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:05:13.547119 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed9eaf0-310e-4056-b10a-f80bbb3ee59c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:05:13.547873 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed9eaf0-310e-4056-b10a-f80bbb3ee59c None admin] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:13.548058 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed9eaf0-310e-4056-b10a-f80bbb3ee59c None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:13.548325 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed9eaf0-310e-4056-b10a-f80bbb3ee59c None admin] PATH_INFO: `/v3/endpoints` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:13.549012 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed9eaf0-310e-4056-b10a-f80bbb3ee59c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:05:13.550373 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed9eaf0-310e-4056-b10a-f80bbb3ee59c None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:05:13.628770 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 491/1002] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 01:05:13 2022] POST /identity/v3/endpoints => generated 325 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:05:13.655077 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1894002f-ef67-4786-957e-ed06b4f65514 None None] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:13.655365 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1894002f-ef67-4786-957e-ed06b4f65514 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:13.655648 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1894002f-ef67-4786-957e-ed06b4f65514 None None] PATH_INFO: `/` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:13.657092 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 512/1003] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:05:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:05:13.663157 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5978a7ca-4815-4b4f-bf39-dc59f71d8248 None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:13.663462 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5978a7ca-4815-4b4f-bf39-dc59f71d8248 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:13.663714 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5978a7ca-4815-4b4f-bf39-dc59f71d8248 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:13.684577 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-5978a7ca-4815-4b4f-bf39-dc59f71d8248 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:05:13.685891 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-5978a7ca-4815-4b4f-bf39-dc59f71d8248 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:05:13.766198 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 492/1004] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:05:13 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 01:05:14.030747 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1894002f-ef67-4786-957e-ed06b4f65514 None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:05:14.135887 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1894002f-ef67-4786-957e-ed06b4f65514 None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:05:14.157547 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:05:14.169539 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fc16dc-ecfd-4a1c-9b41-ab6e06902460 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:05:14.170164 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fc16dc-ecfd-4a1c-9b41-ab6e06902460 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:14.170441 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fc16dc-ecfd-4a1c-9b41-ab6e06902460 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:14.170713 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fc16dc-ecfd-4a1c-9b41-ab6e06902460 None admin] PATH_INFO: `/v3/services/image` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:14.171284 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fc16dc-ecfd-4a1c-9b41-ab6e06902460 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:05:14.172641 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fc16dc-ecfd-4a1c-9b41-ab6e06902460 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:05:14.181966 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application [None req-47fc16dc-ecfd-4a1c-9b41-ab6e06902460 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 31 01:05:14.181966 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:05:14.181966 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 01:05:14.181966 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 01:05:14.181966 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 01:05:14.181966 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 01:05:14.181966 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 01:05:14.181966 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 01:05:14.181966 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 31 01:05:14.181966 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Aug 31 01:05:14.181966 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 01:05:14.181966 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Aug 31 01:05:14.181966 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:05:14.181966 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:05:14.181966 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:05:14.181966 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:05:14.181966 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:05:14.182954 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:05:14.182954 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:05:14.182954 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:05:14.182954 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:05:14.182954 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:05:14.182954 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:05:14.182954 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 01:05:14.182954 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 01:05:14.182954 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 01:05:14.182954 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 01:05:14.182954 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:05:14.182954 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:05:14.182954 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:05:14.182954 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:05:14.182954 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:05:14.182954 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:05:14.183843 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:05:14.183843 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application with Lock( Aug 31 01:05:14.183843 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:05:14.183843 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:05:14.183843 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:05:14.183843 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:05:14.183843 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:05:14.183843 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:05:14.183843 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:05:14.183843 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:05:14.183843 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 01:05:14.183843 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 01:05:14.183843 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 31 01:05:14.183843 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Aug 31 01:05:14.191145 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 513/1005] 10.222.0.20 () {56 vars in 1105 bytes} [Wed Aug 31 01:05:14 2022] GET /identity/v3/services/image => generated 86 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:05:14.191311 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5978a7ca-4815-4b4f-bf39-dc59f71d8248 None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:05:14.280083 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:05:14.287713 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5978a7ca-4815-4b4f-bf39-dc59f71d8248 None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:05:14.321195 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac25fdf1-fa6a-44cb-bedc-96206e4f0250 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:05:14.321796 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac25fdf1-fa6a-44cb-bedc-96206e4f0250 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:14.322074 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac25fdf1-fa6a-44cb-bedc-96206e4f0250 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:14.322345 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac25fdf1-fa6a-44cb-bedc-96206e4f0250 None admin] PATH_INFO: `/v3/services` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:14.322971 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac25fdf1-fa6a-44cb-bedc-96206e4f0250 None admin] RBAC: Adding query filter params (name=image) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:05:14.323270 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac25fdf1-fa6a-44cb-bedc-96206e4f0250 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:05:14.324637 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac25fdf1-fa6a-44cb-bedc-96206e4f0250 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:05:14.329652 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 493/1006] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 01:05:14 2022] GET /identity/v3/services?name=image => generated 122 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:05:14.335584 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fc16dc-ecfd-4a1c-9b41-ab6e06902460 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:05:14.434836 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fc16dc-ecfd-4a1c-9b41-ab6e06902460 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:05:14.468174 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a286d3af-ac99-40cc-aa91-2849b4a25411 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:05:14.468801 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a286d3af-ac99-40cc-aa91-2849b4a25411 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:14.469082 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a286d3af-ac99-40cc-aa91-2849b4a25411 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:14.469352 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a286d3af-ac99-40cc-aa91-2849b4a25411 None admin] PATH_INFO: `/v3/services` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:14.469970 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a286d3af-ac99-40cc-aa91-2849b4a25411 None admin] RBAC: Adding query filter params (type=image) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:05:14.470268 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a286d3af-ac99-40cc-aa91-2849b4a25411 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:05:14.471619 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a286d3af-ac99-40cc-aa91-2849b4a25411 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:05:14.476874 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 514/1007] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 01:05:14 2022] GET /identity/v3/services?type=image => generated 122 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:05:14.936455 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211654b2-a7e0-461f-8820-24d5e0efe223 None None] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:14.936983 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211654b2-a7e0-461f-8820-24d5e0efe223 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:14.937086 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211654b2-a7e0-461f-8820-24d5e0efe223 None None] PATH_INFO: `/` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:14.938533 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 494/1008] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:05:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:05:14.944049 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9c4fd2-a7c8-4462-a3ac-66ee95d270a0 None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:14.944334 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9c4fd2-a7c8-4462-a3ac-66ee95d270a0 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:14.944607 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9c4fd2-a7c8-4462-a3ac-66ee95d270a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:14.963458 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-2a9c4fd2-a7c8-4462-a3ac-66ee95d270a0 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:05:14.964652 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-2a9c4fd2-a7c8-4462-a3ac-66ee95d270a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:05:15.011612 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:05:15.032615 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 515/1009] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:05:14 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 01:05:15.303574 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211654b2-a7e0-461f-8820-24d5e0efe223 None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:05:15.405882 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211654b2-a7e0-461f-8820-24d5e0efe223 None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:05:15.441045 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579e85c5-c1d7-49a9-8b5d-a1fb9af57de9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:05:15.441685 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579e85c5-c1d7-49a9-8b5d-a1fb9af57de9 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:15.441966 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579e85c5-c1d7-49a9-8b5d-a1fb9af57de9 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:15.442239 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579e85c5-c1d7-49a9-8b5d-a1fb9af57de9 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:15.442824 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579e85c5-c1d7-49a9-8b5d-a1fb9af57de9 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:05:15.444236 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579e85c5-c1d7-49a9-8b5d-a1fb9af57de9 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:05:15.453393 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application [None req-579e85c5-c1d7-49a9-8b5d-a1fb9af57de9 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 31 01:05:15.453393 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:05:15.453393 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 01:05:15.453393 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 01:05:15.453393 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 01:05:15.453393 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 01:05:15.453393 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 01:05:15.453393 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 01:05:15.453393 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 31 01:05:15.453393 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Aug 31 01:05:15.453393 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 01:05:15.453393 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Aug 31 01:05:15.453393 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:05:15.453393 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:05:15.453393 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:05:15.453393 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:05:15.453393 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:05:15.454475 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:05:15.454475 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:05:15.454475 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:05:15.454475 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:05:15.454475 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:05:15.454475 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:05:15.454475 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 01:05:15.454475 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 01:05:15.454475 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 01:05:15.454475 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 01:05:15.454475 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:05:15.454475 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:05:15.454475 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:05:15.454475 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:05:15.454475 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:05:15.454475 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:05:15.455333 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:05:15.455333 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application with Lock( Aug 31 01:05:15.455333 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:05:15.455333 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:05:15.455333 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:05:15.455333 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:05:15.455333 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:05:15.455333 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:05:15.455333 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:05:15.455333 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:05:15.455333 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 01:05:15.455333 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 01:05:15.455333 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 31 01:05:15.455333 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Aug 31 01:05:15.458089 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 495/1010] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Aug 31 01:05:15 2022] GET /identity/v3/services/compute => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:05:15.460328 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9c4fd2-a7c8-4462-a3ac-66ee95d270a0 None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:05:15.541018 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d0049d-e9ef-451c-b03c-1165905b5153 None None] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:15.541301 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d0049d-e9ef-451c-b03c-1165905b5153 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:15.541571 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d0049d-e9ef-451c-b03c-1165905b5153 None None] PATH_INFO: `/` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:15.543081 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 496/1011] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:05:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:05:15.548737 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403ee158-3e25-4db0-9954-ae77b3a28e10 None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:15.549087 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403ee158-3e25-4db0-9954-ae77b3a28e10 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:15.549311 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403ee158-3e25-4db0-9954-ae77b3a28e10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:15.562067 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9c4fd2-a7c8-4462-a3ac-66ee95d270a0 None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:05:15.568117 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-403ee158-3e25-4db0-9954-ae77b3a28e10 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:05:15.569368 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-403ee158-3e25-4db0-9954-ae77b3a28e10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:05:15.586016 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:05:15.598544 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df28b8e2-ce66-4573-bfba-dea5e918f91d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:05:15.599217 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df28b8e2-ce66-4573-bfba-dea5e918f91d None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:15.599976 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df28b8e2-ce66-4573-bfba-dea5e918f91d None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:15.600269 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df28b8e2-ce66-4573-bfba-dea5e918f91d None admin] PATH_INFO: `/v3/services` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:15.600953 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df28b8e2-ce66-4573-bfba-dea5e918f91d None admin] RBAC: Adding query filter params (name=compute) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:05:15.601321 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df28b8e2-ce66-4573-bfba-dea5e918f91d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:05:15.602617 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df28b8e2-ce66-4573-bfba-dea5e918f91d None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:05:15.607733 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 516/1012] 10.222.0.20 () {56 vars in 1112 bytes} [Wed Aug 31 01:05:15 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 01:05:15.613180 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df28b8e2-ce66-4573-bfba-dea5e918f91d None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:05:15.641912 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 497/1013] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:05:15 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 01:05:15.658146 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:05:15.711626 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df28b8e2-ce66-4573-bfba-dea5e918f91d None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:05:15.744759 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db185c84-e008-41cf-a4d2-e1d1679dadfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:05:15.745428 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db185c84-e008-41cf-a4d2-e1d1679dadfb None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:15.745771 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db185c84-e008-41cf-a4d2-e1d1679dadfb None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:15.746111 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db185c84-e008-41cf-a4d2-e1d1679dadfb None admin] PATH_INFO: `/v3/services` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:15.746812 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db185c84-e008-41cf-a4d2-e1d1679dadfb None admin] RBAC: Adding query filter params (type=compute) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:05:15.747162 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db185c84-e008-41cf-a4d2-e1d1679dadfb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:05:15.748578 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db185c84-e008-41cf-a4d2-e1d1679dadfb None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:05:15.753459 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 517/1014] 10.222.0.20 () {56 vars in 1112 bytes} [Wed Aug 31 01:05:15 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 01:05:15.759119 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403ee158-3e25-4db0-9954-ae77b3a28e10 None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:05:15.858125 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403ee158-3e25-4db0-9954-ae77b3a28e10 None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:05:15.908191 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db185c84-e008-41cf-a4d2-e1d1679dadfb None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:05:15.920694 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d385a069-fba6-4d1c-a344-f1bd9379088b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:05:15.921336 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d385a069-fba6-4d1c-a344-f1bd9379088b None admin] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:15.921617 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d385a069-fba6-4d1c-a344-f1bd9379088b None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:15.921891 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d385a069-fba6-4d1c-a344-f1bd9379088b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:15.922557 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d385a069-fba6-4d1c-a344-f1bd9379088b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:05:15.923926 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d385a069-fba6-4d1c-a344-f1bd9379088b None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:05:16.001187 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 498/1015] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 01:05:15 2022] POST /identity/v3/endpoints => generated 334 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:05:16.008537 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db185c84-e008-41cf-a4d2-e1d1679dadfb None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:05:16.020560 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:05:16.055862 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b922ef-0ccf-44a9-9185-84847af6385f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:05:16.057138 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b922ef-0ccf-44a9-9185-84847af6385f None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:16.057789 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b922ef-0ccf-44a9-9185-84847af6385f None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:16.058585 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b922ef-0ccf-44a9-9185-84847af6385f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:16.059861 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b922ef-0ccf-44a9-9185-84847af6385f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:05:16.061366 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b922ef-0ccf-44a9-9185-84847af6385f None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:05:16.063188 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 518/1016] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 01:05:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 01:05:16.069138 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d385a069-fba6-4d1c-a344-f1bd9379088b None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:05:16.082248 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:05:16.179751 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d385a069-fba6-4d1c-a344-f1bd9379088b None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:05:16.217039 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823e5c23-b70b-4d35-88b2-281f10d34520 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:05:16.218023 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823e5c23-b70b-4d35-88b2-281f10d34520 None admin] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:16.218307 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823e5c23-b70b-4d35-88b2-281f10d34520 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:16.218581 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823e5c23-b70b-4d35-88b2-281f10d34520 None admin] PATH_INFO: `/v3/projects` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:16.219222 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823e5c23-b70b-4d35-88b2-281f10d34520 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:05:16.220657 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823e5c23-b70b-4d35-88b2-281f10d34520 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:05:16.293332 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 499/1017] 10.222.0.20 () {60 vars in 1141 bytes} [Wed Aug 31 01:05:16 2022] POST /identity/v3/projects => generated 311 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:05:16.417028 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136e99b4-604d-4419-b2fb-75a29d973b4a None None] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:16.419509 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136e99b4-604d-4419-b2fb-75a29d973b4a None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:16.419804 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136e99b4-604d-4419-b2fb-75a29d973b4a None None] PATH_INFO: `/` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:16.421323 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 519/1018] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:05:16 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:05:16.427093 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43aae68-c69d-4f45-ad4d-6ae53e3dd0aa None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:16.427383 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43aae68-c69d-4f45-ad4d-6ae53e3dd0aa None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:16.427658 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43aae68-c69d-4f45-ad4d-6ae53e3dd0aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:16.446903 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-f43aae68-c69d-4f45-ad4d-6ae53e3dd0aa None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:05:16.448021 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-f43aae68-c69d-4f45-ad4d-6ae53e3dd0aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:05:16.510249 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:05:16.521473 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 500/1019] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:05:16 2022] POST /identity/v3/auth/tokens => generated 1873 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 01:05:16.789428 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136e99b4-604d-4419-b2fb-75a29d973b4a None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:05:16.895368 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136e99b4-604d-4419-b2fb-75a29d973b4a None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:05:16.934418 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b22d8e9-8258-4642-bd4c-309c22982b3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:05:16.935024 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b22d8e9-8258-4642-bd4c-309c22982b3c None admin] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:16.935304 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b22d8e9-8258-4642-bd4c-309c22982b3c None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:16.935576 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b22d8e9-8258-4642-bd4c-309c22982b3c None admin] PATH_INFO: `/v3/services` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:16.936239 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b22d8e9-8258-4642-bd4c-309c22982b3c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:05:16.937700 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b22d8e9-8258-4642-bd4c-309c22982b3c None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:05:17.035375 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 520/1020] 10.222.0.20 () {60 vars in 1141 bytes} [Wed Aug 31 01:05:16 2022] POST /identity/v3/services => generated 242 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:05:18.015267 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85df38a-4b1f-49ae-8e48-d1c833e6f82c None None] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:18.015774 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85df38a-4b1f-49ae-8e48-d1c833e6f82c None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:18.015852 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85df38a-4b1f-49ae-8e48-d1c833e6f82c None None] PATH_INFO: `/` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:18.017346 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 501/1021] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:05:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:05:18.022221 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc00814-7be4-4c09-88ad-ccc3aacddc47 None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:18.022610 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc00814-7be4-4c09-88ad-ccc3aacddc47 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:18.022771 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc00814-7be4-4c09-88ad-ccc3aacddc47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:18.040899 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-8dc00814-7be4-4c09-88ad-ccc3aacddc47 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:05:18.042098 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-8dc00814-7be4-4c09-88ad-ccc3aacddc47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:05:18.068031 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:05:18.120836 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 521/1022] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:05:18 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 01:05:18.226484 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dafc9f-e35b-44ac-a3d9-530d51a66c95 None None] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:18.226797 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dafc9f-e35b-44ac-a3d9-530d51a66c95 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:18.227041 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dafc9f-e35b-44ac-a3d9-530d51a66c95 None None] PATH_INFO: `/` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:18.228505 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 502/1023] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:05:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:05:18.233533 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d38ceec-b7f5-4490-96df-1213a4ff9bae None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:18.233827 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d38ceec-b7f5-4490-96df-1213a4ff9bae None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:18.234099 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d38ceec-b7f5-4490-96df-1213a4ff9bae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:18.252346 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-0d38ceec-b7f5-4490-96df-1213a4ff9bae None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:05:18.253552 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-0d38ceec-b7f5-4490-96df-1213a4ff9bae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:05:18.317667 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 522/1024] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:05:18 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 01:05:18.393628 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1dafc9f-e35b-44ac-a3d9-530d51a66c95 None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:05:18.497061 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1dafc9f-e35b-44ac-a3d9-530d51a66c95 None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:05:18.535325 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7da6cc7-5e05-49f1-80f6-5e0d28611c3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:05:18.535960 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7da6cc7-5e05-49f1-80f6-5e0d28611c3e None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:18.536246 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7da6cc7-5e05-49f1-80f6-5e0d28611c3e None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:18.536524 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7da6cc7-5e05-49f1-80f6-5e0d28611c3e None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:18.547264 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7da6cc7-5e05-49f1-80f6-5e0d28611c3e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 01:05:18.552111 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application [None req-f7da6cc7-5e05-49f1-80f6-5e0d28611c3e None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 31 01:05:18.552111 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:05:18.552111 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:05:18.552111 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:05:18.552111 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:05:18.552111 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:05:18.552111 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:05:18.552111 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:05:18.552111 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:05:18.552111 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:05:18.552111 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:05:18.552111 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:05:18.552111 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 01:05:18.552111 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 01:05:18.552111 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 01:05:18.552111 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 01:05:18.553790 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 01:05:18.553790 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 01:05:18.553790 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 01:05:18.553790 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 01:05:18.553790 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:05:18.553790 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:05:18.553790 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:05:18.553790 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:05:18.553790 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:05:18.553790 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:05:18.553790 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:05:18.553790 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application with Lock( Aug 31 01:05:18.553790 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:05:18.553790 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:05:18.553790 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:05:18.553790 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:05:18.554667 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:05:18.554667 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:05:18.554667 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:05:18.554667 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:05:18.554667 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 01:05:18.554667 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 01:05:18.554667 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 01:05:18.554667 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 01:05:18.554667 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 01:05:18.554667 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 01:05:18.554667 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 31 01:05:18.554667 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Aug 31 01:05:18.557063 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 503/1025] 10.222.0.20 () {56 vars in 1120 bytes} [Wed Aug 31 01:05:18 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:05:18.559699 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d38ceec-b7f5-4490-96df-1213a4ff9bae None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:05:18.589512 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7da6cc7-5e05-49f1-80f6-5e0d28611c3e None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:05:18.643399 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:05:18.664190 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:05:18.676166 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d38ceec-b7f5-4490-96df-1213a4ff9bae None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:05:18.694485 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7da6cc7-5e05-49f1-80f6-5e0d28611c3e None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:05:18.712229 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a44f81-d188-43e7-ba6d-9c4f7e6c339c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:05:18.712880 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a44f81-d188-43e7-ba6d-9c4f7e6c339c None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:18.713160 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a44f81-d188-43e7-ba6d-9c4f7e6c339c None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:18.713441 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a44f81-d188-43e7-ba6d-9c4f7e6c339c None admin] PATH_INFO: `/v3/roles` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:18.714113 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a44f81-d188-43e7-ba6d-9c4f7e6c339c None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:05:18.714419 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a44f81-d188-43e7-ba6d-9c4f7e6c339c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:05:18.715786 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a44f81-d188-43e7-ba6d-9c4f7e6c339c None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:05:18.723751 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 523/1026] 10.222.0.20 () {56 vars in 1115 bytes} [Wed Aug 31 01:05:18 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:05:18.730282 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a44f81-d188-43e7-ba6d-9c4f7e6c339c None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:05:18.738064 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcce6b6e-7d10-457d-a290-c5a0704474a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:05:18.738957 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcce6b6e-7d10-457d-a290-c5a0704474a0 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:18.739418 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcce6b6e-7d10-457d-a290-c5a0704474a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:18.739959 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcce6b6e-7d10-457d-a290-c5a0704474a0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:18.741519 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcce6b6e-7d10-457d-a290-c5a0704474a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:05:18.743266 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcce6b6e-7d10-457d-a290-c5a0704474a0 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:05:18.745548 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 504/1027] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 01:05:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 01:05:18.750731 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcce6b6e-7d10-457d-a290-c5a0704474a0 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:05:18.840907 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a44f81-d188-43e7-ba6d-9c4f7e6c339c None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:05:18.859241 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcce6b6e-7d10-457d-a290-c5a0704474a0 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:05:18.877217 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:05:18.882533 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0196d2de-3b01-4446-99e8-6ae2fbf01b8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:05:18.883173 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0196d2de-3b01-4446-99e8-6ae2fbf01b8e None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:18.883760 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0196d2de-3b01-4446-99e8-6ae2fbf01b8e None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:18.883760 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0196d2de-3b01-4446-99e8-6ae2fbf01b8e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:18.897663 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be64a05b-0c8e-4256-bee2-e1e6e2fa1c33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:05:18.898349 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be64a05b-0c8e-4256-bee2-e1e6e2fa1c33 None admin] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:18.898545 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be64a05b-0c8e-4256-bee2-e1e6e2fa1c33 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:18.898831 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be64a05b-0c8e-4256-bee2-e1e6e2fa1c33 None admin] PATH_INFO: `/v3/users` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:18.898950 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0196d2de-3b01-4446-99e8-6ae2fbf01b8e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:05:18.899905 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be64a05b-0c8e-4256-bee2-e1e6e2fa1c33 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:05:18.900480 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0196d2de-3b01-4446-99e8-6ae2fbf01b8e None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:05:18.900927 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be64a05b-0c8e-4256-bee2-e1e6e2fa1c33 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:05:18.916604 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application [None req-0196d2de-3b01-4446-99e8-6ae2fbf01b8e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 01:05:18.916604 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:05:18.916604 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:05:18.916604 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:05:18.916604 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:05:18.916604 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:05:18.916604 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:05:18.916604 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:05:18.916604 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:05:18.916604 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:05:18.916604 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:05:18.916604 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:05:18.916604 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 01:05:18.916604 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 01:05:18.916604 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 01:05:18.916604 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 01:05:18.917673 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:05:18.917673 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:05:18.917673 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:05:18.917673 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:05:18.917673 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:05:18.917673 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:05:18.917673 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:05:18.917673 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application with Lock( Aug 31 01:05:18.917673 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:05:18.917673 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:05:18.917673 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:05:18.917673 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:05:18.917673 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:05:18.917673 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:05:18.917673 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:05:18.917673 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:05:18.917673 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 01:05:18.923721 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 01:05:18.923721 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 01:05:18.923721 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Aug 31 01:05:18.923721 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 524/1028] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 01:05:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:05:18.923948 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0196d2de-3b01-4446-99e8-6ae2fbf01b8e None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:05:18.926680 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:05:19.012065 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 505/1029] 10.222.0.20 () {60 vars in 1132 bytes} [Wed Aug 31 01:05:18 2022] POST /identity/v3/users => generated 285 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 01:05:19.013420 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f588e0-aef0-4e41-8240-c2a795774f85 None None] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:19.013709 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f588e0-aef0-4e41-8240-c2a795774f85 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:19.013977 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f588e0-aef0-4e41-8240-c2a795774f85 None None] PATH_INFO: `/` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:19.015360 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 506/1030] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:05:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:05:19.021098 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb8211b-4964-4d36-b81e-674783fe558c None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:19.021965 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb8211b-4964-4d36-b81e-674783fe558c None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:19.022244 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb8211b-4964-4d36-b81e-674783fe558c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:19.024439 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0196d2de-3b01-4446-99e8-6ae2fbf01b8e None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:05:19.042190 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-cbb8211b-4964-4d36-b81e-674783fe558c None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:05:19.043525 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-cbb8211b-4964-4d36-b81e-674783fe558c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:05:19.062874 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af00d39b-f51c-4c5b-8f23-c896c828159a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:05:19.063510 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af00d39b-f51c-4c5b-8f23-c896c828159a None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:19.063794 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af00d39b-f51c-4c5b-8f23-c896c828159a None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:19.064069 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af00d39b-f51c-4c5b-8f23-c896c828159a None admin] PATH_INFO: `/v3/domains` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:19.064760 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af00d39b-f51c-4c5b-8f23-c896c828159a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:05:19.065065 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af00d39b-f51c-4c5b-8f23-c896c828159a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:05:19.066505 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af00d39b-f51c-4c5b-8f23-c896c828159a None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:05:19.086130 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 525/1031] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 31 01:05:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:05:19.088074 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af00d39b-f51c-4c5b-8f23-c896c828159a None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:05:19.143728 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 507/1032] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:05:19 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 01:05:19.175208 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:05:19.214483 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af00d39b-f51c-4c5b-8f23-c896c828159a None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:05:19.249108 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aba6359-e0db-418d-a503-c84ff8edc06c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:05:19.249996 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aba6359-e0db-418d-a503-c84ff8edc06c None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:19.250280 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aba6359-e0db-418d-a503-c84ff8edc06c None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:19.250556 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aba6359-e0db-418d-a503-c84ff8edc06c None admin] PATH_INFO: `/v3/users/nova` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:19.258193 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aba6359-e0db-418d-a503-c84ff8edc06c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:05:19.259661 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aba6359-e0db-418d-a503-c84ff8edc06c None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:05:19.272218 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application [None req-4aba6359-e0db-418d-a503-c84ff8edc06c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 31 01:05:19.272218 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:05:19.272218 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:05:19.272218 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:05:19.272218 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:05:19.272218 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:05:19.272218 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:05:19.272218 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:05:19.272218 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:05:19.272218 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:05:19.272218 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:05:19.272218 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:05:19.272218 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 01:05:19.272218 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 01:05:19.272218 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 01:05:19.272218 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 01:05:19.273418 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:05:19.273418 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:05:19.273418 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 01:05:19.273418 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 01:05:19.273418 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 01:05:19.273418 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 01:05:19.273418 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:05:19.273418 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:05:19.273418 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:05:19.273418 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:05:19.273418 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:05:19.273418 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application with Lock( Aug 31 01:05:19.273418 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:05:19.273418 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:05:19.273418 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:05:19.273418 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:05:19.273418 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:05:19.273418 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:05:19.274547 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:05:19.274547 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:05:19.274547 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 01:05:19.274547 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 01:05:19.274547 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 01:05:19.274547 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 01:05:19.274547 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 01:05:19.274547 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 01:05:19.274547 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 31 01:05:19.274547 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Aug 31 01:05:19.274547 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 526/1033] 10.222.0.20 () {56 vars in 1093 bytes} [Wed Aug 31 01:05:19 2022] GET /identity/v3/users/nova => generated 82 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 01:05:19.280926 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb8211b-4964-4d36-b81e-674783fe558c None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:05:19.381443 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb8211b-4964-4d36-b81e-674783fe558c None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:05:19.383838 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:05:19.415533 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aba6359-e0db-418d-a503-c84ff8edc06c None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:05:19.416073 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fceb5d2e-af0a-46cb-a85b-22092a80f97a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:05:19.416710 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fceb5d2e-af0a-46cb-a85b-22092a80f97a None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:19.417018 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fceb5d2e-af0a-46cb-a85b-22092a80f97a None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:19.417297 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fceb5d2e-af0a-46cb-a85b-22092a80f97a None admin] PATH_INFO: `/v3/users` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:19.418039 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fceb5d2e-af0a-46cb-a85b-22092a80f97a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:05:19.418345 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fceb5d2e-af0a-46cb-a85b-22092a80f97a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:05:19.419793 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fceb5d2e-af0a-46cb-a85b-22092a80f97a None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:05:19.439518 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 508/1034] 10.222.0.20 () {56 vars in 1133 bytes} [Wed Aug 31 01:05:19 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:05:19.444781 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fceb5d2e-af0a-46cb-a85b-22092a80f97a None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:05:19.523644 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aba6359-e0db-418d-a503-c84ff8edc06c None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:05:19.545056 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:05:19.556816 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fceb5d2e-af0a-46cb-a85b-22092a80f97a None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:05:19.556985 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536f66d1-785d-4d9c-a897-388a458204b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:05:19.557573 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536f66d1-785d-4d9c-a897-388a458204b5 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:19.557851 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536f66d1-785d-4d9c-a897-388a458204b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:19.558294 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536f66d1-785d-4d9c-a897-388a458204b5 None admin] PATH_INFO: `/v3/services/image` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:19.558771 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536f66d1-785d-4d9c-a897-388a458204b5 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:05:19.560144 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536f66d1-785d-4d9c-a897-388a458204b5 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:05:19.569140 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application [None req-536f66d1-785d-4d9c-a897-388a458204b5 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 31 01:05:19.569140 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:05:19.569140 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 01:05:19.569140 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 01:05:19.569140 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 01:05:19.569140 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 01:05:19.569140 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 01:05:19.569140 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 01:05:19.569140 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 31 01:05:19.569140 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Aug 31 01:05:19.569140 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 01:05:19.569140 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Aug 31 01:05:19.569140 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:05:19.569140 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:05:19.569140 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:05:19.569140 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:05:19.569140 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:05:19.570380 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:05:19.570380 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:05:19.570380 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:05:19.570380 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:05:19.570380 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:05:19.570380 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:05:19.570380 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 01:05:19.570380 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 01:05:19.570380 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 01:05:19.570380 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 01:05:19.570380 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:05:19.570380 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:05:19.570380 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:05:19.570380 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:05:19.570380 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:05:19.570380 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:05:19.571405 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:05:19.571405 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application with Lock( Aug 31 01:05:19.571405 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:05:19.571405 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:05:19.571405 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:05:19.571405 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:05:19.571405 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:05:19.571405 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:05:19.571405 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:05:19.571405 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:05:19.571405 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 01:05:19.571405 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 01:05:19.571405 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 31 01:05:19.571405 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Aug 31 01:05:19.571405 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 527/1035] 10.222.0.20 () {56 vars in 1105 bytes} [Wed Aug 31 01:05:19 2022] GET /identity/v3/services/image => generated 86 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 01:05:19.576404 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536f66d1-785d-4d9c-a897-388a458204b5 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:05:19.598200 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4316538-8d96-47d9-b7a1-812234881114 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:05:19.598821 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4316538-8d96-47d9-b7a1-812234881114 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:19.599105 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4316538-8d96-47d9-b7a1-812234881114 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:19.599983 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4316538-8d96-47d9-b7a1-812234881114 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:19.630555 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:05:19.670185 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4316538-8d96-47d9-b7a1-812234881114 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:05:19.671798 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4316538-8d96-47d9-b7a1-812234881114 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:05:19.691999 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536f66d1-785d-4d9c-a897-388a458204b5 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:05:19.730844 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0611300b-924e-479b-a44c-38d3a333af03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:05:19.731549 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0611300b-924e-479b-a44c-38d3a333af03 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:19.731905 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0611300b-924e-479b-a44c-38d3a333af03 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:19.732253 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0611300b-924e-479b-a44c-38d3a333af03 None admin] PATH_INFO: `/v3/services` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:19.732985 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0611300b-924e-479b-a44c-38d3a333af03 None admin] RBAC: Adding query filter params (name=image) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:05:19.733297 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0611300b-924e-479b-a44c-38d3a333af03 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:05:19.734654 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0611300b-924e-479b-a44c-38d3a333af03 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:05:19.739619 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 528/1036] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 01:05:19 2022] GET /identity/v3/services?name=image => generated 122 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 01:05:19.745891 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0611300b-924e-479b-a44c-38d3a333af03 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:05:19.777997 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 509/1037] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Aug 31 01:05:19 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 01:05:19.783221 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4316538-8d96-47d9-b7a1-812234881114 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:05:19.806109 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:05:19.848408 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:05:19.865658 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0611300b-924e-479b-a44c-38d3a333af03 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:05:19.888463 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4316538-8d96-47d9-b7a1-812234881114 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:05:19.904802 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e58d1bc-420b-4aab-beec-ce8b34325bc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:05:19.905521 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e58d1bc-420b-4aab-beec-ce8b34325bc5 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:19.905914 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e58d1bc-420b-4aab-beec-ce8b34325bc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:19.906264 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e58d1bc-420b-4aab-beec-ce8b34325bc5 None admin] PATH_INFO: `/v3/services` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:19.906967 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e58d1bc-420b-4aab-beec-ce8b34325bc5 None admin] RBAC: Adding query filter params (type=image) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:05:19.907344 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e58d1bc-420b-4aab-beec-ce8b34325bc5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:05:19.908811 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e58d1bc-420b-4aab-beec-ce8b34325bc5 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:05:19.914313 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 529/1038] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 01:05:19 2022] GET /identity/v3/services?type=image => generated 350 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 01:05:19.920077 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e58d1bc-420b-4aab-beec-ce8b34325bc5 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:05:19.936497 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd74a029-acb5-476a-bcb5-4e0c73e482b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:05:19.937178 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd74a029-acb5-476a-bcb5-4e0c73e482b0 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:19.937477 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd74a029-acb5-476a-bcb5-4e0c73e482b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:19.937873 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd74a029-acb5-476a-bcb5-4e0c73e482b0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:19.949214 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd74a029-acb5-476a-bcb5-4e0c73e482b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:05:19.950927 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd74a029-acb5-476a-bcb5-4e0c73e482b0 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:05:19.966076 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application [None req-cd74a029-acb5-476a-bcb5-4e0c73e482b0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 01:05:19.966076 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:05:19.966076 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:05:19.966076 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:05:19.966076 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:05:19.966076 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:05:19.966076 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:05:19.966076 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:05:19.966076 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:05:19.966076 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:05:19.966076 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:05:19.966076 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:05:19.966076 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 01:05:19.966076 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 01:05:19.966076 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 01:05:19.966076 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 01:05:19.967220 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:05:19.967220 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:05:19.967220 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:05:19.967220 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:05:19.967220 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:05:19.967220 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:05:19.967220 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:05:19.967220 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application with Lock( Aug 31 01:05:19.967220 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:05:19.967220 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:05:19.967220 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:05:19.967220 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:05:19.967220 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:05:19.967220 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:05:19.967220 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:05:19.967220 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:05:19.967220 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 01:05:19.968251 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 01:05:19.968251 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 01:05:19.968251 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Aug 31 01:05:19.968251 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 510/1039] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 01:05:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 01:05:19.972657 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd74a029-acb5-476a-bcb5-4e0c73e482b0 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:05:20.026016 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e58d1bc-420b-4aab-beec-ce8b34325bc5 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:05:20.041757 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:05:20.064477 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1fba5c-d483-4628-8646-a2a0baffc8d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:05:20.065357 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1fba5c-d483-4628-8646-a2a0baffc8d0 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:20.065807 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1fba5c-d483-4628-8646-a2a0baffc8d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:20.066506 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1fba5c-d483-4628-8646-a2a0baffc8d0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:20.066785 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1fba5c-d483-4628-8646-a2a0baffc8d0 None admin] RBAC: Adding query filter params (interface=public, service_id=4c5f0b158aaf441d915686b4eadb3145, region_id=RegionOne) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:05:20.067077 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1fba5c-d483-4628-8646-a2a0baffc8d0 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:05:20.068500 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1fba5c-d483-4628-8646-a2a0baffc8d0 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:05:20.078647 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 530/1040] 10.222.0.20 () {56 vars in 1251 bytes} [Wed Aug 31 01:05:19 2022] GET /identity/v3/endpoints?service_id=4c5f0b158aaf441d915686b4eadb3145&interface=public®ion_id=RegionOne => generated 194 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 01:05:20.079453 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1fba5c-d483-4628-8646-a2a0baffc8d0 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:05:20.080434 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd74a029-acb5-476a-bcb5-4e0c73e482b0 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:05:20.096969 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:05:20.118878 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39e00ff-7214-4100-ba2f-735774c334d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:05:20.119487 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39e00ff-7214-4100-ba2f-735774c334d0 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:20.119761 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39e00ff-7214-4100-ba2f-735774c334d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:20.120028 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39e00ff-7214-4100-ba2f-735774c334d0 None admin] PATH_INFO: `/v3/domains` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:20.120671 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39e00ff-7214-4100-ba2f-735774c334d0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:05:20.121001 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39e00ff-7214-4100-ba2f-735774c334d0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:05:20.122375 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39e00ff-7214-4100-ba2f-735774c334d0 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:05:20.136032 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 511/1041] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 31 01:05:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 01:05:20.141717 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39e00ff-7214-4100-ba2f-735774c334d0 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:05:20.194193 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1fba5c-d483-4628-8646-a2a0baffc8d0 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:05:20.232484 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aafd0f8-535c-49e1-91b0-72059ae64d56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:05:20.233223 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aafd0f8-535c-49e1-91b0-72059ae64d56 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:20.233505 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aafd0f8-535c-49e1-91b0-72059ae64d56 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:20.233776 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aafd0f8-535c-49e1-91b0-72059ae64d56 None admin] PATH_INFO: `/v3/services` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:20.234370 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aafd0f8-535c-49e1-91b0-72059ae64d56 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:05:20.235829 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aafd0f8-535c-49e1-91b0-72059ae64d56 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:05:20.242369 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 531/1042] 10.222.0.20 () {56 vars in 1087 bytes} [Wed Aug 31 01:05:20 2022] GET /identity/v3/services => generated 1957 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 01:05:20.259766 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39e00ff-7214-4100-ba2f-735774c334d0 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:05:20.299102 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b67659-1cd9-45a5-8327-04fb10a2962d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:05:20.300003 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b67659-1cd9-45a5-8327-04fb10a2962d None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:20.300288 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b67659-1cd9-45a5-8327-04fb10a2962d None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:20.300566 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b67659-1cd9-45a5-8327-04fb10a2962d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:20.308819 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b67659-1cd9-45a5-8327-04fb10a2962d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:05:20.310301 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b67659-1cd9-45a5-8327-04fb10a2962d None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:05:20.323195 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application [None req-65b67659-1cd9-45a5-8327-04fb10a2962d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 01:05:20.323195 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:05:20.323195 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:05:20.323195 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:05:20.323195 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:05:20.323195 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:05:20.323195 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:05:20.323195 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:05:20.323195 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:05:20.323195 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:05:20.323195 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:05:20.323195 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:05:20.323195 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 01:05:20.323195 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 01:05:20.323195 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 01:05:20.324164 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 01:05:20.324164 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:05:20.324164 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:05:20.324164 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:05:20.324164 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:05:20.324164 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:05:20.324164 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:05:20.324164 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:05:20.324164 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application with Lock( Aug 31 01:05:20.324164 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:05:20.324164 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:05:20.324164 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:05:20.324164 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:05:20.324164 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:05:20.324164 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:05:20.324164 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:05:20.324164 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:05:20.325612 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 01:05:20.325612 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 01:05:20.325612 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 01:05:20.325612 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 01:05:20.325612 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 01:05:20.325612 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 01:05:20.325612 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 01:05:20.325612 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Aug 31 01:05:20.325612 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 512/1043] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Aug 31 01:05:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 01:05:20.330473 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aafd0f8-535c-49e1-91b0-72059ae64d56 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:05:20.378403 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:05:20.436629 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aafd0f8-535c-49e1-91b0-72059ae64d56 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:05:20.470755 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a2dac6-706d-4f0c-a848-4c64aea080aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:05:20.471364 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a2dac6-706d-4f0c-a848-4c64aea080aa None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:20.471646 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a2dac6-706d-4f0c-a848-4c64aea080aa None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:20.471918 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a2dac6-706d-4f0c-a848-4c64aea080aa None admin] PATH_INFO: `/v3/projects` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:20.472609 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a2dac6-706d-4f0c-a848-4c64aea080aa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:05:20.472942 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a2dac6-706d-4f0c-a848-4c64aea080aa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:05:20.474918 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a2dac6-706d-4f0c-a848-4c64aea080aa None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:05:20.486507 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 532/1044] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Aug 31 01:05:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:05:20.492790 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b67659-1cd9-45a5-8327-04fb10a2962d None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:05:20.527901 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:05:20.595291 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b67659-1cd9-45a5-8327-04fb10a2962d None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:05:20.631085 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2612d0-da4d-4952-a066-83c0a9541716 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:05:20.631839 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2612d0-da4d-4952-a066-83c0a9541716 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:20.632219 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2612d0-da4d-4952-a066-83c0a9541716 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:20.632592 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2612d0-da4d-4952-a066-83c0a9541716 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:20.633445 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2612d0-da4d-4952-a066-83c0a9541716 None admin] RBAC: Adding query filter params (role.id=586b679002e645439ed44fe1ca4a7cc6, scope.project.id=60b2abc15c2849ba8a1ed92117cc981b, user.id=8744c9de803048f583d0317d01f6a540) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:05:20.633874 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2612d0-da4d-4952-a066-83c0a9541716 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:05:20.635398 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2612d0-da4d-4952-a066-83c0a9541716 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:05:20.648760 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 513/1045] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Aug 31 01:05:20 2022] GET /identity/v3/role_assignments?user.id=8744c9de803048f583d0317d01f6a540&scope.project.id=60b2abc15c2849ba8a1ed92117cc981b&role.id=586b679002e645439ed44fe1ca4a7cc6 => generated 259 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 01:05:21.017070 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3343ec-6333-40a9-9772-b0f98d3ff948 None None] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:21.017423 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3343ec-6333-40a9-9772-b0f98d3ff948 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:21.017633 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3343ec-6333-40a9-9772-b0f98d3ff948 None None] PATH_INFO: `/` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:21.019022 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 533/1046] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:05:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:05:21.024635 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d129b61b-15a6-449c-87fa-50de05e0a38f None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:21.025107 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d129b61b-15a6-449c-87fa-50de05e0a38f None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:21.025394 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d129b61b-15a6-449c-87fa-50de05e0a38f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:21.044134 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-d129b61b-15a6-449c-87fa-50de05e0a38f None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:05:21.045352 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-d129b61b-15a6-449c-87fa-50de05e0a38f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:05:21.114926 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 514/1047] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:05:21 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 01:05:21.384427 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3343ec-6333-40a9-9772-b0f98d3ff948 None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:05:21.485576 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3343ec-6333-40a9-9772-b0f98d3ff948 None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:05:21.487101 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:05:21.519170 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77697967-1084-421f-b7f3-b2183693e332 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:05:21.519795 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77697967-1084-421f-b7f3-b2183693e332 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:21.520076 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77697967-1084-421f-b7f3-b2183693e332 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:21.520347 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77697967-1084-421f-b7f3-b2183693e332 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:21.529974 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77697967-1084-421f-b7f3-b2183693e332 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 01:05:21.535123 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application [None req-77697967-1084-421f-b7f3-b2183693e332 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 01:05:21.535123 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:05:21.535123 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:05:21.535123 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:05:21.535123 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:05:21.535123 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:05:21.535123 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:05:21.535123 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:05:21.535123 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:05:21.535123 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:05:21.535123 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:05:21.535123 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:05:21.535123 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 01:05:21.535123 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 01:05:21.535123 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 01:05:21.535123 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 01:05:21.536054 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 01:05:21.536054 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 01:05:21.536054 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 01:05:21.536054 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 01:05:21.536054 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:05:21.536054 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:05:21.536054 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:05:21.536054 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:05:21.536054 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:05:21.536054 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:05:21.536054 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:05:21.536054 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application with Lock( Aug 31 01:05:21.536054 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:05:21.536054 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:05:21.536054 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:05:21.536054 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:05:21.537445 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:05:21.537445 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:05:21.537445 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:05:21.537445 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:05:21.537445 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 01:05:21.537445 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 01:05:21.537445 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 01:05:21.537445 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 01:05:21.537445 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 01:05:21.537445 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 01:05:21.537445 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 01:05:21.537445 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Aug 31 01:05:21.541042 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 534/1048] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Aug 31 01:05:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:05:21.542615 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d129b61b-15a6-449c-87fa-50de05e0a38f None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:05:21.570493 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:05:21.642656 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d129b61b-15a6-449c-87fa-50de05e0a38f None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:05:21.676085 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30922fa2-1dbc-48c8-a176-820e11a385f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:05:21.676704 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30922fa2-1dbc-48c8-a176-820e11a385f5 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:21.677027 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30922fa2-1dbc-48c8-a176-820e11a385f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:21.677920 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30922fa2-1dbc-48c8-a176-820e11a385f5 None admin] PATH_INFO: `/v3/roles` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:21.678546 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30922fa2-1dbc-48c8-a176-820e11a385f5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:05:21.678855 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30922fa2-1dbc-48c8-a176-820e11a385f5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:05:21.680276 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30922fa2-1dbc-48c8-a176-820e11a385f5 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:05:21.694889 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 515/1049] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Aug 31 01:05:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 01:05:21.696387 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77697967-1084-421f-b7f3-b2183693e332 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:05:21.799318 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77697967-1084-421f-b7f3-b2183693e332 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:05:21.834471 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738bc08b-76f7-491b-b7fa-6c36880712f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:05:21.839457 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738bc08b-76f7-491b-b7fa-6c36880712f8 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:21.839457 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738bc08b-76f7-491b-b7fa-6c36880712f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:21.839457 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738bc08b-76f7-491b-b7fa-6c36880712f8 None admin] PATH_INFO: `/v3/users/68c5f345cd89497b905b9863997efeff` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:21.849389 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738bc08b-76f7-491b-b7fa-6c36880712f8 None admin] RBAC: Authorizing `identity:get_user(user_id=68c5f345cd89497b905b9863997efeff)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:05:21.850823 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738bc08b-76f7-491b-b7fa-6c36880712f8 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:05:21.857057 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 535/1050] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Aug 31 01:05:21 2022] GET /identity/v3/users/68c5f345cd89497b905b9863997efeff => generated 285 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:05:21.858449 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30922fa2-1dbc-48c8-a176-820e11a385f5 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:05:21.959754 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:05:21.969484 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30922fa2-1dbc-48c8-a176-820e11a385f5 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:05:22.003798 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bd0550-6f7f-43e5-9834-30fe9251af46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:05:22.004413 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bd0550-6f7f-43e5-9834-30fe9251af46 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:22.004699 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bd0550-6f7f-43e5-9834-30fe9251af46 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:22.005006 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bd0550-6f7f-43e5-9834-30fe9251af46 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:22.072664 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bd0550-6f7f-43e5-9834-30fe9251af46 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:05:22.074150 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bd0550-6f7f-43e5-9834-30fe9251af46 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:05:22.162559 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:05:22.176818 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 516/1051] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Aug 31 01:05:21 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 01:05:22.185254 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c42052-683f-46ef-b5f0-cd1d22f7bb47 None None] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:22.185254 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c42052-683f-46ef-b5f0-cd1d22f7bb47 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:22.185254 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c42052-683f-46ef-b5f0-cd1d22f7bb47 None None] PATH_INFO: `/` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:22.185254 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 536/1052] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:05:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:05:22.185605 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bd0550-6f7f-43e5-9834-30fe9251af46 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:05:22.185778 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d73276f-4a46-41b8-897d-282f5358b95b None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:22.185778 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d73276f-4a46-41b8-897d-282f5358b95b None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:22.185778 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d73276f-4a46-41b8-897d-282f5358b95b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:22.204088 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-6d73276f-4a46-41b8-897d-282f5358b95b None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:05:22.205525 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-6d73276f-4a46-41b8-897d-282f5358b95b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:05:22.221256 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:05:22.280547 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bd0550-6f7f-43e5-9834-30fe9251af46 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:05:22.284855 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 537/1053] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:05:22 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 01:05:22.325863 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d805d2-b374-4028-bab0-f9d1f8588061 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:05:22.326774 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d805d2-b374-4028-bab0-f9d1f8588061 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:22.327270 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d805d2-b374-4028-bab0-f9d1f8588061 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:22.327729 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d805d2-b374-4028-bab0-f9d1f8588061 None admin] PATH_INFO: `/v3/projects/daa87f11c6484ca0bcfbf25b676324fe` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:22.329090 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d805d2-b374-4028-bab0-f9d1f8588061 None admin] RBAC: Authorizing `identity:get_project(project_id=daa87f11c6484ca0bcfbf25b676324fe)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:05:22.330772 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d805d2-b374-4028-bab0-f9d1f8588061 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:05:22.332958 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 517/1054] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Aug 31 01:05:22 2022] GET /identity/v3/projects/daa87f11c6484ca0bcfbf25b676324fe => generated 311 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 01:05:22.338738 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d73276f-4a46-41b8-897d-282f5358b95b None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:05:22.440385 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d73276f-4a46-41b8-897d-282f5358b95b None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:05:22.474791 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de24b96-5f61-4286-94d7-c059f35de954 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:05:22.475402 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de24b96-5f61-4286-94d7-c059f35de954 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:22.475687 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de24b96-5f61-4286-94d7-c059f35de954 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:22.475962 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de24b96-5f61-4286-94d7-c059f35de954 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:22.476679 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de24b96-5f61-4286-94d7-c059f35de954 None admin] RBAC: Adding query filter params (role.id=d009411496b8452b88e77ffb8db015a7, scope.project.id=daa87f11c6484ca0bcfbf25b676324fe, user.id=68c5f345cd89497b905b9863997efeff) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:05:22.477010 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de24b96-5f61-4286-94d7-c059f35de954 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:05:22.478506 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de24b96-5f61-4286-94d7-c059f35de954 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:05:22.489031 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 538/1055] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Aug 31 01:05:22 2022] GET /identity/v3/role_assignments?user.id=68c5f345cd89497b905b9863997efeff&scope.project.id=daa87f11c6484ca0bcfbf25b676324fe&role.id=d009411496b8452b88e77ffb8db015a7 => generated 259 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 01:05:22.511284 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee7c319-5f90-47cb-8c42-a34a33e396d7 None None] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:22.511584 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee7c319-5f90-47cb-8c42-a34a33e396d7 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:22.511852 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee7c319-5f90-47cb-8c42-a34a33e396d7 None None] PATH_INFO: `/` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:22.513302 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 518/1056] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:05:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:05:22.519081 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f9da63-ac48-4f7e-8e6e-1f22336fff33 None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:22.519400 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f9da63-ac48-4f7e-8e6e-1f22336fff33 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:22.519639 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f9da63-ac48-4f7e-8e6e-1f22336fff33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:22.539748 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-54f9da63-ac48-4f7e-8e6e-1f22336fff33 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:05:22.541003 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-54f9da63-ac48-4f7e-8e6e-1f22336fff33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:05:22.552276 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:05:22.562185 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee7c319-5f90-47cb-8c42-a34a33e396d7 None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:05:22.615734 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 539/1057] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:05:22 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 01:05:22.631831 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:05:22.676118 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee7c319-5f90-47cb-8c42-a34a33e396d7 None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:05:22.710392 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff942c3c-47e1-4f09-be41-7e5cd501e792 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:05:22.710935 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff942c3c-47e1-4f09-be41-7e5cd501e792 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:22.711211 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff942c3c-47e1-4f09-be41-7e5cd501e792 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:22.711479 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff942c3c-47e1-4f09-be41-7e5cd501e792 None admin] PATH_INFO: `/v3/services/image` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:22.712049 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff942c3c-47e1-4f09-be41-7e5cd501e792 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:05:22.713434 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff942c3c-47e1-4f09-be41-7e5cd501e792 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:05:22.722160 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application [None req-ff942c3c-47e1-4f09-be41-7e5cd501e792 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 31 01:05:22.722160 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:05:22.722160 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 01:05:22.722160 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 01:05:22.722160 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 01:05:22.722160 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 01:05:22.722160 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 01:05:22.722160 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 01:05:22.722160 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 31 01:05:22.722160 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Aug 31 01:05:22.722160 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 01:05:22.722160 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Aug 31 01:05:22.722160 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:05:22.722160 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:05:22.722160 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:05:22.722160 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:05:22.722160 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:05:22.723128 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:05:22.723128 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:05:22.723128 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:05:22.723128 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:05:22.723128 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:05:22.723128 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:05:22.723128 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 01:05:22.723128 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 01:05:22.723128 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 01:05:22.723128 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 01:05:22.723128 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:05:22.723128 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:05:22.723128 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:05:22.723128 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:05:22.723128 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:05:22.723128 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:05:22.729705 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:05:22.729705 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application with Lock( Aug 31 01:05:22.729705 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:05:22.729705 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:05:22.729705 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:05:22.729705 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:05:22.729705 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:05:22.729705 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:05:22.729705 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:05:22.729705 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:05:22.729705 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 01:05:22.729705 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 01:05:22.729705 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 31 01:05:22.729705 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Aug 31 01:05:22.729705 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 519/1058] 10.222.0.20 () {56 vars in 1105 bytes} [Wed Aug 31 01:05:22 2022] GET /identity/v3/services/image => generated 86 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:05:22.730544 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f9da63-ac48-4f7e-8e6e-1f22336fff33 None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:05:22.832362 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f9da63-ac48-4f7e-8e6e-1f22336fff33 None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:05:22.867576 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb58d880-b271-4cb1-adf0-fabb3d763a8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:05:22.868181 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb58d880-b271-4cb1-adf0-fabb3d763a8d None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:22.868457 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb58d880-b271-4cb1-adf0-fabb3d763a8d None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:22.868750 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb58d880-b271-4cb1-adf0-fabb3d763a8d None admin] PATH_INFO: `/v3/services` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:22.869359 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb58d880-b271-4cb1-adf0-fabb3d763a8d None admin] RBAC: Adding query filter params (name=image) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:05:22.869656 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb58d880-b271-4cb1-adf0-fabb3d763a8d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:05:22.871003 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb58d880-b271-4cb1-adf0-fabb3d763a8d None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:05:22.880822 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 540/1059] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 01:05:22 2022] GET /identity/v3/services?name=image => generated 122 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:05:22.882108 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff942c3c-47e1-4f09-be41-7e5cd501e792 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:05:22.882630 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb58d880-b271-4cb1-adf0-fabb3d763a8d None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:05:22.902791 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:05:22.988774 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb58d880-b271-4cb1-adf0-fabb3d763a8d None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:05:22.993197 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff942c3c-47e1-4f09-be41-7e5cd501e792 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:05:23.017783 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:05:23.026071 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33082e71-d7e3-436e-8dcf-3873308b9189 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:05:23.030359 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33082e71-d7e3-436e-8dcf-3873308b9189 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:23.030359 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33082e71-d7e3-436e-8dcf-3873308b9189 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:23.030359 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33082e71-d7e3-436e-8dcf-3873308b9189 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:23.030573 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4212dbb-9a53-4342-9774-4a3fdc9cea05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:05:23.030573 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4212dbb-9a53-4342-9774-4a3fdc9cea05 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:23.030573 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4212dbb-9a53-4342-9774-4a3fdc9cea05 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:23.030573 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4212dbb-9a53-4342-9774-4a3fdc9cea05 None admin] PATH_INFO: `/v3/services` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:23.031008 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4212dbb-9a53-4342-9774-4a3fdc9cea05 None admin] RBAC: Adding query filter params (type=image) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:05:23.031283 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4212dbb-9a53-4342-9774-4a3fdc9cea05 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:05:23.032653 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4212dbb-9a53-4342-9774-4a3fdc9cea05 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:05:23.038198 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 520/1060] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 01:05:22 2022] GET /identity/v3/services?type=image => generated 350 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 01:05:23.038335 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33082e71-d7e3-436e-8dcf-3873308b9189 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 01:05:23.043260 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application [None req-33082e71-d7e3-436e-8dcf-3873308b9189 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 31 01:05:23.043260 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:05:23.043260 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:05:23.043260 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:05:23.043260 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:05:23.043260 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:05:23.043260 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:05:23.043260 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:05:23.043260 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:05:23.043260 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:05:23.043260 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:05:23.043260 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:05:23.043260 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 01:05:23.043260 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 01:05:23.043260 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 01:05:23.043260 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 01:05:23.044173 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 01:05:23.044173 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 01:05:23.044173 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 01:05:23.044173 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 01:05:23.044173 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:05:23.044173 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:05:23.044173 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:05:23.044173 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:05:23.044173 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:05:23.044173 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:05:23.044173 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:05:23.044173 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application with Lock( Aug 31 01:05:23.044173 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:05:23.044173 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:05:23.044173 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:05:23.044173 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:05:23.045687 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:05:23.045687 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:05:23.045687 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:05:23.045687 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:05:23.045687 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 01:05:23.045687 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 01:05:23.045687 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 01:05:23.045687 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 01:05:23.045687 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 01:05:23.045687 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 01:05:23.045687 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 31 01:05:23.045687 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Aug 31 01:05:23.046496 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4212dbb-9a53-4342-9774-4a3fdc9cea05 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:05:23.046589 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 541/1061] 10.222.0.20 () {56 vars in 1120 bytes} [Wed Aug 31 01:05:22 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 01:05:23.050036 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33082e71-d7e3-436e-8dcf-3873308b9189 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:05:23.118870 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:05:23.153565 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33082e71-d7e3-436e-8dcf-3873308b9189 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:05:23.161481 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4212dbb-9a53-4342-9774-4a3fdc9cea05 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:05:23.190790 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8515c18-3808-4bd9-8be0-12835ba06a84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:05:23.191402 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8515c18-3808-4bd9-8be0-12835ba06a84 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:23.191690 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8515c18-3808-4bd9-8be0-12835ba06a84 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:23.191968 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8515c18-3808-4bd9-8be0-12835ba06a84 None admin] PATH_INFO: `/v3/roles` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:23.192631 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8515c18-3808-4bd9-8be0-12835ba06a84 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:05:23.192982 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8515c18-3808-4bd9-8be0-12835ba06a84 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:05:23.194422 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8515c18-3808-4bd9-8be0-12835ba06a84 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:05:23.197916 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f72a9f-566a-40e8-925a-27642e8d2d40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:05:23.198531 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f72a9f-566a-40e8-925a-27642e8d2d40 None admin] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:23.198815 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f72a9f-566a-40e8-925a-27642e8d2d40 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:23.199092 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f72a9f-566a-40e8-925a-27642e8d2d40 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:23.200008 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f72a9f-566a-40e8-925a-27642e8d2d40 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:05:23.201386 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f72a9f-566a-40e8-925a-27642e8d2d40 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:05:23.202798 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 542/1062] 10.222.0.20 () {56 vars in 1115 bytes} [Wed Aug 31 01:05:23 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 01:05:23.208183 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8515c18-3808-4bd9-8be0-12835ba06a84 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:05:23.285241 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 521/1063] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 01:05:23 2022] POST /identity/v3/endpoints => generated 327 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 01:05:23.310427 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8515c18-3808-4bd9-8be0-12835ba06a84 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:05:23.342230 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:05:23.356701 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e58aa2-3b57-482a-8a33-080d9a7639b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:05:23.357379 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e58aa2-3b57-482a-8a33-080d9a7639b3 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:23.357659 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e58aa2-3b57-482a-8a33-080d9a7639b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:23.357930 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e58aa2-3b57-482a-8a33-080d9a7639b3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:23.369470 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e58aa2-3b57-482a-8a33-080d9a7639b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:05:23.371186 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e58aa2-3b57-482a-8a33-080d9a7639b3 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:05:23.386684 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application [None req-f7e58aa2-3b57-482a-8a33-080d9a7639b3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 01:05:23.386684 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:05:23.386684 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:05:23.386684 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:05:23.386684 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:05:23.386684 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:05:23.386684 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:05:23.386684 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:05:23.386684 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:05:23.386684 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:05:23.386684 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:05:23.386684 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:05:23.386684 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 01:05:23.386684 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 01:05:23.386684 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 01:05:23.386684 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 01:05:23.387765 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:05:23.387765 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:05:23.387765 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:05:23.387765 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:05:23.387765 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:05:23.387765 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:05:23.387765 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:05:23.387765 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application with Lock( Aug 31 01:05:23.387765 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:05:23.387765 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:05:23.387765 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:05:23.387765 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:05:23.387765 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:05:23.387765 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:05:23.387765 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:05:23.387765 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:05:23.387765 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 01:05:23.388806 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 01:05:23.388806 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 01:05:23.388806 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Aug 31 01:05:23.388806 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 543/1064] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 01:05:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 01:05:23.393748 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f72a9f-566a-40e8-925a-27642e8d2d40 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:05:23.443893 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:05:23.505002 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f72a9f-566a-40e8-925a-27642e8d2d40 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:05:23.540824 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8ffae3-2ee3-4ae4-ab03-ecbf42d29aba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:05:23.541796 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8ffae3-2ee3-4ae4-ab03-ecbf42d29aba None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:23.542281 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8ffae3-2ee3-4ae4-ab03-ecbf42d29aba None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:23.542759 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8ffae3-2ee3-4ae4-ab03-ecbf42d29aba None admin] PATH_INFO: `/v3/domains` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:23.543597 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8ffae3-2ee3-4ae4-ab03-ecbf42d29aba None admin] RBAC: Adding query filter params (name=Default) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:05:23.544118 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8ffae3-2ee3-4ae4-ab03-ecbf42d29aba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:05:23.545726 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8ffae3-2ee3-4ae4-ab03-ecbf42d29aba None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:05:23.558622 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 522/1065] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 31 01:05:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:05:23.563879 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e58aa2-3b57-482a-8a33-080d9a7639b3 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:05:23.666941 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e58aa2-3b57-482a-8a33-080d9a7639b3 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:05:23.700414 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e421ba-3c91-4059-bf64-d3163249c597 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:05:23.701067 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e421ba-3c91-4059-bf64-d3163249c597 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:23.701348 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e421ba-3c91-4059-bf64-d3163249c597 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:23.701619 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e421ba-3c91-4059-bf64-d3163249c597 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:23.708428 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e421ba-3c91-4059-bf64-d3163249c597 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:05:23.710414 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e421ba-3c91-4059-bf64-d3163249c597 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:05:23.721702 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application [None req-59e421ba-3c91-4059-bf64-d3163249c597 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 31 01:05:23.721702 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:05:23.721702 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:05:23.721702 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:05:23.721702 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:05:23.721702 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:05:23.721702 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:05:23.721702 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:05:23.721702 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:05:23.721702 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:05:23.721702 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:05:23.721702 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:05:23.721702 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 01:05:23.721702 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 01:05:23.721702 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 01:05:23.721702 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 01:05:23.722648 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:05:23.722648 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:05:23.722648 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 01:05:23.722648 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 01:05:23.722648 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 01:05:23.722648 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 01:05:23.722648 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:05:23.722648 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:05:23.722648 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:05:23.722648 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:05:23.722648 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:05:23.722648 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application with Lock( Aug 31 01:05:23.722648 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:05:23.722648 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:05:23.722648 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:05:23.722648 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:05:23.722648 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:05:23.722648 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:05:23.723613 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:05:23.723613 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:05:23.723613 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 01:05:23.723613 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 01:05:23.723613 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 01:05:23.723613 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 01:05:23.723613 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 01:05:23.723613 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 01:05:23.723613 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 31 01:05:23.723613 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Aug 31 01:05:23.723613 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 544/1066] 10.222.0.20 () {56 vars in 1093 bytes} [Wed Aug 31 01:05:23 2022] GET /identity/v3/users/nova => generated 82 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:05:23.728482 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8ffae3-2ee3-4ae4-ab03-ecbf42d29aba None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:05:23.830690 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8ffae3-2ee3-4ae4-ab03-ecbf42d29aba None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:05:23.832159 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:05:23.866571 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638ffd9d-6daa-453c-8925-f5b727280881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:05:23.867215 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638ffd9d-6daa-453c-8925-f5b727280881 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:23.867502 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638ffd9d-6daa-453c-8925-f5b727280881 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:23.867778 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638ffd9d-6daa-453c-8925-f5b727280881 None admin] PATH_INFO: `/v3/users` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:23.868504 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638ffd9d-6daa-453c-8925-f5b727280881 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:05:23.868840 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638ffd9d-6daa-453c-8925-f5b727280881 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:05:23.870268 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638ffd9d-6daa-453c-8925-f5b727280881 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:05:23.892262 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 523/1067] 10.222.0.20 () {56 vars in 1133 bytes} [Wed Aug 31 01:05:23 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:05:23.893858 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e421ba-3c91-4059-bf64-d3163249c597 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:05:23.937482 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:05:23.998324 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e421ba-3c91-4059-bf64-d3163249c597 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:05:24.032404 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0ad236-1a85-4265-bb2f-85f26cc65de4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:05:24.033066 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0ad236-1a85-4265-bb2f-85f26cc65de4 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:24.033353 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0ad236-1a85-4265-bb2f-85f26cc65de4 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:24.033628 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0ad236-1a85-4265-bb2f-85f26cc65de4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:24.044622 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0ad236-1a85-4265-bb2f-85f26cc65de4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:05:24.046106 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0ad236-1a85-4265-bb2f-85f26cc65de4 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:05:24.060028 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application [None req-3f0ad236-1a85-4265-bb2f-85f26cc65de4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 01:05:24.060028 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:05:24.060028 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:05:24.060028 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:05:24.060028 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:05:24.060028 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:05:24.060028 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:05:24.060028 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:05:24.060028 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:05:24.060028 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:05:24.060028 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:05:24.060028 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:05:24.060028 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 01:05:24.060028 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 01:05:24.060028 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 01:05:24.060028 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 01:05:24.061021 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:05:24.061021 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:05:24.061021 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:05:24.061021 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:05:24.061021 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:05:24.061021 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:05:24.061021 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:05:24.061021 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application with Lock( Aug 31 01:05:24.061021 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:05:24.061021 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:05:24.061021 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:05:24.061021 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:05:24.061021 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:05:24.061021 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:05:24.061021 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:05:24.061021 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:05:24.061021 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 01:05:24.061969 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 01:05:24.061969 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 01:05:24.061969 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Aug 31 01:05:24.061969 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 545/1068] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 01:05:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:05:24.066900 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638ffd9d-6daa-453c-8925-f5b727280881 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:05:24.169769 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638ffd9d-6daa-453c-8925-f5b727280881 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:05:24.203922 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74e1548-f9fa-4393-8f3a-790879e71e8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:05:24.204535 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74e1548-f9fa-4393-8f3a-790879e71e8d None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:24.204816 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74e1548-f9fa-4393-8f3a-790879e71e8d None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:24.205098 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74e1548-f9fa-4393-8f3a-790879e71e8d None admin] PATH_INFO: `/v3/domains` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:24.205740 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74e1548-f9fa-4393-8f3a-790879e71e8d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:05:24.206047 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74e1548-f9fa-4393-8f3a-790879e71e8d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:05:24.207422 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74e1548-f9fa-4393-8f3a-790879e71e8d None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:05:24.223787 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 524/1069] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 31 01:05:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:05:24.225703 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0ad236-1a85-4265-bb2f-85f26cc65de4 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:05:24.317278 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:05:24.326903 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0ad236-1a85-4265-bb2f-85f26cc65de4 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:05:24.360961 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b588bb-8008-471a-b90e-fc70ce4da3f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:05:24.361590 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b588bb-8008-471a-b90e-fc70ce4da3f3 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:24.361869 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b588bb-8008-471a-b90e-fc70ce4da3f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:24.362143 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b588bb-8008-471a-b90e-fc70ce4da3f3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:24.368920 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b588bb-8008-471a-b90e-fc70ce4da3f3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:05:24.370397 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b588bb-8008-471a-b90e-fc70ce4da3f3 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:05:24.384039 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application [None req-65b588bb-8008-471a-b90e-fc70ce4da3f3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 01:05:24.384039 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:05:24.384039 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:05:24.384039 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:05:24.384039 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:05:24.384039 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:05:24.384039 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:05:24.384039 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:05:24.384039 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:05:24.384039 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:05:24.384039 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:05:24.384039 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:05:24.384039 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 01:05:24.384039 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 01:05:24.384039 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 01:05:24.385038 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 01:05:24.385038 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:05:24.385038 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:05:24.385038 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:05:24.385038 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:05:24.385038 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:05:24.385038 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:05:24.385038 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:05:24.385038 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application with Lock( Aug 31 01:05:24.385038 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:05:24.385038 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:05:24.385038 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:05:24.385038 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:05:24.385038 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:05:24.385038 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:05:24.385038 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:05:24.385038 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:05:24.385940 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 01:05:24.385940 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 01:05:24.385940 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 01:05:24.385940 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 01:05:24.385940 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 01:05:24.385940 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 01:05:24.385940 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 01:05:24.385940 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Aug 31 01:05:24.388799 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 546/1070] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Aug 31 01:05:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:05:24.391139 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74e1548-f9fa-4393-8f3a-790879e71e8d None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:05:24.414060 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:05:24.437218 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66213d2-0963-4c29-a7a5-ea49bb48af47 None None] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:24.437458 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66213d2-0963-4c29-a7a5-ea49bb48af47 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:24.437712 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66213d2-0963-4c29-a7a5-ea49bb48af47 None None] PATH_INFO: `/` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:24.439051 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 547/1071] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:05:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:05:24.444371 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e8a90d-7f57-466b-88e7-0e40bcb98e92 None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:24.444651 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e8a90d-7f57-466b-88e7-0e40bcb98e92 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:24.444930 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e8a90d-7f57-466b-88e7-0e40bcb98e92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:24.463563 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-80e8a90d-7f57-466b-88e7-0e40bcb98e92 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:05:24.464773 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-80e8a90d-7f57-466b-88e7-0e40bcb98e92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:05:24.494249 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74e1548-f9fa-4393-8f3a-790879e71e8d None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:05:24.530050 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47342fd-63dc-4320-94ba-9c502ab4c335 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:05:24.530690 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47342fd-63dc-4320-94ba-9c502ab4c335 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:24.530976 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47342fd-63dc-4320-94ba-9c502ab4c335 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:24.531259 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47342fd-63dc-4320-94ba-9c502ab4c335 None admin] PATH_INFO: `/v3/projects` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:24.531943 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47342fd-63dc-4320-94ba-9c502ab4c335 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:05:24.532253 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47342fd-63dc-4320-94ba-9c502ab4c335 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:05:24.533720 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47342fd-63dc-4320-94ba-9c502ab4c335 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:05:24.540033 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 548/1072] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:05:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 01:05:24.550059 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 525/1073] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Aug 31 01:05:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:05:24.555926 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e8a90d-7f57-466b-88e7-0e40bcb98e92 None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:05:24.637035 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:05:24.655807 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e8a90d-7f57-466b-88e7-0e40bcb98e92 None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:05:24.690871 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c07c6a-4bad-426a-998b-b670987d4c9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:05:24.691464 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c07c6a-4bad-426a-998b-b670987d4c9c None admin] REQUEST_METHOD: `PUT` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:24.691746 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c07c6a-4bad-426a-998b-b670987d4c9c None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:24.692022 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c07c6a-4bad-426a-998b-b670987d4c9c None admin] PATH_INFO: `/v3/projects/60b2abc15c2849ba8a1ed92117cc981b/users/8744c9de803048f583d0317d01f6a540/roles/586b679002e645439ed44fe1ca4a7cc6` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:24.694802 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c07c6a-4bad-426a-998b-b670987d4c9c None admin] RBAC: Authorizing `identity:create_grant(project_id=60b2abc15c2849ba8a1ed92117cc981b, user_id=8744c9de803048f583d0317d01f6a540, role_id=586b679002e645439ed44fe1ca4a7cc6)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:05:24.696232 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c07c6a-4bad-426a-998b-b670987d4c9c None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:05:24.771687 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 549/1074] 10.222.0.20 () {58 vars in 1439 bytes} [Wed Aug 31 01:05:24 2022] PUT /identity/v3/projects/60b2abc15c2849ba8a1ed92117cc981b/users/8744c9de803048f583d0317d01f6a540/roles/586b679002e645439ed44fe1ca4a7cc6 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 01:05:24.812106 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47342fd-63dc-4320-94ba-9c502ab4c335 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:05:24.910577 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:05:24.931365 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47342fd-63dc-4320-94ba-9c502ab4c335 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:05:24.967162 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d45d7b-f555-4eba-897a-79b6abd19da3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:05:24.967719 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d45d7b-f555-4eba-897a-79b6abd19da3 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:24.967999 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d45d7b-f555-4eba-897a-79b6abd19da3 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:24.968275 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d45d7b-f555-4eba-897a-79b6abd19da3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:24.978646 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d45d7b-f555-4eba-897a-79b6abd19da3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 01:05:24.983475 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application [None req-06d45d7b-f555-4eba-897a-79b6abd19da3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 01:05:24.983475 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:05:24.983475 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:05:24.983475 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:05:24.983475 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:05:24.983475 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:05:24.983475 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:05:24.983475 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:05:24.983475 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:05:24.983475 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:05:24.983475 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:05:24.983475 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:05:24.983475 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 01:05:24.983475 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 01:05:24.983475 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 01:05:24.983475 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 01:05:24.984378 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 01:05:24.984378 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 01:05:24.984378 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 01:05:24.984378 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 01:05:24.984378 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:05:24.984378 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:05:24.984378 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:05:24.984378 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:05:24.984378 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:05:24.984378 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:05:24.984378 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:05:24.984378 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application with Lock( Aug 31 01:05:24.984378 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:05:24.984378 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:05:24.984378 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:05:24.984378 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:05:24.989468 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:05:24.989468 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:05:24.989468 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:05:24.989468 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:05:24.989468 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 01:05:24.989468 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 01:05:24.989468 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 01:05:24.989468 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 01:05:24.989468 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 01:05:24.989468 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 01:05:24.989468 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 01:05:24.989468 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Aug 31 01:05:24.989468 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 526/1075] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Aug 31 01:05:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:05:24.991592 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c07c6a-4bad-426a-998b-b670987d4c9c None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:05:25.092734 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c07c6a-4bad-426a-998b-b670987d4c9c None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:05:25.129027 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753f0fb7-93ea-4d46-9ee6-edcd997108a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:05:25.129647 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753f0fb7-93ea-4d46-9ee6-edcd997108a4 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:25.129930 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753f0fb7-93ea-4d46-9ee6-edcd997108a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:25.130231 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753f0fb7-93ea-4d46-9ee6-edcd997108a4 None admin] PATH_INFO: `/v3/roles` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:25.130907 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753f0fb7-93ea-4d46-9ee6-edcd997108a4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:05:25.131213 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753f0fb7-93ea-4d46-9ee6-edcd997108a4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:05:25.132572 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753f0fb7-93ea-4d46-9ee6-edcd997108a4 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:05:25.145439 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 550/1076] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Aug 31 01:05:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:05:25.146997 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d45d7b-f555-4eba-897a-79b6abd19da3 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:05:25.223324 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793c163a-622f-4641-b3ea-55b09380d628 None None] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:25.223506 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793c163a-622f-4641-b3ea-55b09380d628 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:25.223787 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793c163a-622f-4641-b3ea-55b09380d628 None None] PATH_INFO: `/` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:25.225699 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 551/1077] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:05:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 01:05:25.230469 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3263d6e7-05ea-423d-88da-f4dac19d670f None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:25.230754 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3263d6e7-05ea-423d-88da-f4dac19d670f None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:25.231032 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3263d6e7-05ea-423d-88da-f4dac19d670f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:25.237602 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:05:25.250252 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-3263d6e7-05ea-423d-88da-f4dac19d670f None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:05:25.251325 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-3263d6e7-05ea-423d-88da-f4dac19d670f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:05:25.253845 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d45d7b-f555-4eba-897a-79b6abd19da3 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:05:25.289012 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4c4fc1-8bc6-4019-8781-6b0cb014dec7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:05:25.289643 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4c4fc1-8bc6-4019-8781-6b0cb014dec7 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:25.289978 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4c4fc1-8bc6-4019-8781-6b0cb014dec7 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:25.290253 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4c4fc1-8bc6-4019-8781-6b0cb014dec7 None admin] PATH_INFO: `/v3/users/68c5f345cd89497b905b9863997efeff` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:25.291408 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4c4fc1-8bc6-4019-8781-6b0cb014dec7 None admin] RBAC: Authorizing `identity:get_user(user_id=68c5f345cd89497b905b9863997efeff)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:05:25.292887 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4c4fc1-8bc6-4019-8781-6b0cb014dec7 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:05:25.299294 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 527/1078] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Aug 31 01:05:25 2022] GET /identity/v3/users/68c5f345cd89497b905b9863997efeff => generated 285 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:05:25.301047 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4c4fc1-8bc6-4019-8781-6b0cb014dec7 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:05:25.315718 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:05:25.324623 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 552/1079] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:05:25 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 01:05:25.409989 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4c4fc1-8bc6-4019-8781-6b0cb014dec7 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:05:25.445986 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8a7abc-a479-4b22-b206-e0bf9def6369 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:05:25.446879 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8a7abc-a479-4b22-b206-e0bf9def6369 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:25.447169 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8a7abc-a479-4b22-b206-e0bf9def6369 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:25.447447 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8a7abc-a479-4b22-b206-e0bf9def6369 None admin] PATH_INFO: `/v3/projects/daa87f11c6484ca0bcfbf25b676324fe` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:25.448597 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8a7abc-a479-4b22-b206-e0bf9def6369 None admin] RBAC: Authorizing `identity:get_project(project_id=daa87f11c6484ca0bcfbf25b676324fe)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:05:25.450398 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8a7abc-a479-4b22-b206-e0bf9def6369 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:05:25.456900 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 528/1080] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Aug 31 01:05:25 2022] GET /identity/v3/projects/daa87f11c6484ca0bcfbf25b676324fe => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 01:05:25.459023 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3263d6e7-05ea-423d-88da-f4dac19d670f None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:05:25.558551 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3263d6e7-05ea-423d-88da-f4dac19d670f None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:05:25.587626 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:05:25.591553 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8a7abc-a479-4b22-b206-e0bf9def6369 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:05:25.594341 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2a6239-c2ed-4111-908b-e4b6fbd10352 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:05:25.595004 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2a6239-c2ed-4111-908b-e4b6fbd10352 None admin] REQUEST_METHOD: `PUT` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:25.595349 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2a6239-c2ed-4111-908b-e4b6fbd10352 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:25.599614 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2a6239-c2ed-4111-908b-e4b6fbd10352 None admin] PATH_INFO: `/v3/projects/daa87f11c6484ca0bcfbf25b676324fe/users/68c5f345cd89497b905b9863997efeff/roles/d009411496b8452b88e77ffb8db015a7` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:25.599614 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2a6239-c2ed-4111-908b-e4b6fbd10352 None admin] RBAC: Authorizing `identity:create_grant(project_id=daa87f11c6484ca0bcfbf25b676324fe, user_id=68c5f345cd89497b905b9863997efeff, role_id=d009411496b8452b88e77ffb8db015a7)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:05:25.599614 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2a6239-c2ed-4111-908b-e4b6fbd10352 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:05:25.669612 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:05:25.675464 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 553/1081] 10.222.0.20 () {58 vars in 1439 bytes} [Wed Aug 31 01:05:25 2022] PUT /identity/v3/projects/daa87f11c6484ca0bcfbf25b676324fe/users/68c5f345cd89497b905b9863997efeff/roles/d009411496b8452b88e77ffb8db015a7 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 01:05:25.694535 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8a7abc-a479-4b22-b206-e0bf9def6369 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:05:25.734700 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131c6055-3172-4809-9c86-b28bf257f2bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:05:25.735328 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131c6055-3172-4809-9c86-b28bf257f2bb None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:25.735616 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131c6055-3172-4809-9c86-b28bf257f2bb None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:25.735893 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131c6055-3172-4809-9c86-b28bf257f2bb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:25.809285 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131c6055-3172-4809-9c86-b28bf257f2bb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:05:25.810678 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131c6055-3172-4809-9c86-b28bf257f2bb None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:05:25.898398 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:05:25.923097 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 529/1082] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Aug 31 01:05:25 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 31 01:05:25.929042 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2a6239-c2ed-4111-908b-e4b6fbd10352 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:05:26.030231 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2a6239-c2ed-4111-908b-e4b6fbd10352 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:05:26.064334 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdad02b-d867-443a-92f8-78fb107721f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:05:26.064917 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdad02b-d867-443a-92f8-78fb107721f4 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:26.065202 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdad02b-d867-443a-92f8-78fb107721f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:26.065480 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdad02b-d867-443a-92f8-78fb107721f4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:26.076171 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdad02b-d867-443a-92f8-78fb107721f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:05:26.077638 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdad02b-d867-443a-92f8-78fb107721f4 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:05:26.092682 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application [None req-9bdad02b-d867-443a-92f8-78fb107721f4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 01:05:26.092682 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:05:26.092682 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:05:26.092682 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:05:26.092682 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:05:26.092682 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:05:26.092682 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:05:26.092682 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:05:26.092682 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:05:26.092682 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:05:26.092682 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:05:26.092682 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:05:26.092682 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 01:05:26.092682 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 01:05:26.092682 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 01:05:26.092682 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 01:05:26.093652 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:05:26.093652 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:05:26.093652 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:05:26.093652 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:05:26.093652 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:05:26.093652 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:05:26.093652 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:05:26.093652 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application with Lock( Aug 31 01:05:26.093652 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:05:26.093652 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:05:26.093652 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:05:26.093652 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:05:26.093652 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:05:26.093652 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:05:26.093652 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:05:26.093652 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:05:26.093652 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 01:05:26.094567 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 01:05:26.094567 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 01:05:26.094567 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Aug 31 01:05:26.094567 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 554/1083] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 01:05:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 01:05:26.100660 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131c6055-3172-4809-9c86-b28bf257f2bb None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:05:26.202174 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131c6055-3172-4809-9c86-b28bf257f2bb None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:05:26.236509 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921af082-586c-41ee-a407-97ddda1f10fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:05:26.237144 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921af082-586c-41ee-a407-97ddda1f10fa None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:26.237429 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921af082-586c-41ee-a407-97ddda1f10fa None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:26.237709 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921af082-586c-41ee-a407-97ddda1f10fa None admin] PATH_INFO: `/v3/domains` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:26.238355 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921af082-586c-41ee-a407-97ddda1f10fa None admin] RBAC: Adding query filter params (name=Default) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:05:26.238665 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921af082-586c-41ee-a407-97ddda1f10fa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:05:26.240569 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921af082-586c-41ee-a407-97ddda1f10fa None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:05:26.252029 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 530/1084] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 31 01:05:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:05:26.685870 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b12c54-1146-4a6c-a760-02e4ac7380b2 None None] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:26.686384 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b12c54-1146-4a6c-a760-02e4ac7380b2 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:26.686460 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b12c54-1146-4a6c-a760-02e4ac7380b2 None None] PATH_INFO: `/` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:26.687799 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 555/1085] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:05:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:05:26.692965 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b0fc26-0f55-43ab-8cbf-9e5170af1e3d None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:26.693365 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b0fc26-0f55-43ab-8cbf-9e5170af1e3d None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:26.693635 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b0fc26-0f55-43ab-8cbf-9e5170af1e3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:26.711863 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-90b0fc26-0f55-43ab-8cbf-9e5170af1e3d None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:05:26.713075 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-90b0fc26-0f55-43ab-8cbf-9e5170af1e3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:05:26.741488 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:05:26.780447 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 531/1086] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:05:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 01:05:27.047551 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b12c54-1146-4a6c-a760-02e4ac7380b2 None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:05:27.094392 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:05:27.149718 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b12c54-1146-4a6c-a760-02e4ac7380b2 None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:05:27.182599 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c903ec42-5090-43d3-bc1b-6c304f42ec86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:05:27.183226 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c903ec42-5090-43d3-bc1b-6c304f42ec86 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:27.183510 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c903ec42-5090-43d3-bc1b-6c304f42ec86 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:27.183787 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c903ec42-5090-43d3-bc1b-6c304f42ec86 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:27.194146 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c903ec42-5090-43d3-bc1b-6c304f42ec86 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 01:05:27.199011 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application [None req-c903ec42-5090-43d3-bc1b-6c304f42ec86 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 31 01:05:27.199011 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:05:27.199011 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:05:27.199011 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:05:27.199011 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:05:27.199011 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:05:27.199011 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:05:27.199011 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:05:27.199011 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:05:27.199011 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:05:27.199011 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:05:27.199011 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:05:27.199011 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 01:05:27.199011 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 01:05:27.199011 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 01:05:27.199011 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 01:05:27.200336 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 01:05:27.200336 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 01:05:27.200336 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 01:05:27.200336 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 01:05:27.200336 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:05:27.200336 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:05:27.200336 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:05:27.200336 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:05:27.200336 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:05:27.200336 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:05:27.200336 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:05:27.200336 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application with Lock( Aug 31 01:05:27.200336 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:05:27.200336 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:05:27.200336 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:05:27.200336 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:05:27.201613 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:05:27.201613 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:05:27.201613 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:05:27.201613 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:05:27.201613 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 01:05:27.201613 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 01:05:27.201613 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 01:05:27.201613 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 01:05:27.201613 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 01:05:27.201613 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 01:05:27.201613 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 31 01:05:27.201613 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Aug 31 01:05:27.201613 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 556/1087] 10.222.0.20 () {56 vars in 1120 bytes} [Wed Aug 31 01:05:27 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:05:27.206356 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b0fc26-0f55-43ab-8cbf-9e5170af1e3d None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:05:27.309287 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b0fc26-0f55-43ab-8cbf-9e5170af1e3d None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:05:27.343101 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416dc185-488b-45b4-b0f0-ff8fd65c4a0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:05:27.343708 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416dc185-488b-45b4-b0f0-ff8fd65c4a0f None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:27.343991 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416dc185-488b-45b4-b0f0-ff8fd65c4a0f None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:27.344270 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416dc185-488b-45b4-b0f0-ff8fd65c4a0f None admin] PATH_INFO: `/v3/roles` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:27.344965 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416dc185-488b-45b4-b0f0-ff8fd65c4a0f None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:05:27.345276 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416dc185-488b-45b4-b0f0-ff8fd65c4a0f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:05:27.346658 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416dc185-488b-45b4-b0f0-ff8fd65c4a0f None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:05:27.359321 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 532/1088] 10.222.0.20 () {56 vars in 1115 bytes} [Wed Aug 31 01:05:27 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:05:27.360507 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c903ec42-5090-43d3-bc1b-6c304f42ec86 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:05:27.456515 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c903ec42-5090-43d3-bc1b-6c304f42ec86 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:05:27.459376 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:05:27.489173 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcdf34f-7dab-43f5-9ab7-7e5ffd86d43b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:05:27.489822 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcdf34f-7dab-43f5-9ab7-7e5ffd86d43b None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:27.490100 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcdf34f-7dab-43f5-9ab7-7e5ffd86d43b None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:27.490372 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcdf34f-7dab-43f5-9ab7-7e5ffd86d43b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:27.500463 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdcdf34f-7dab-43f5-9ab7-7e5ffd86d43b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:05:27.502464 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdcdf34f-7dab-43f5-9ab7-7e5ffd86d43b None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:05:27.517183 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application [None req-fdcdf34f-7dab-43f5-9ab7-7e5ffd86d43b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 01:05:27.517183 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:05:27.517183 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:05:27.517183 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:05:27.517183 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:05:27.517183 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:05:27.517183 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:05:27.517183 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:05:27.517183 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:05:27.517183 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:05:27.517183 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:05:27.517183 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:05:27.517183 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 01:05:27.517183 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 01:05:27.517183 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 01:05:27.517183 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 01:05:27.518238 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:05:27.518238 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:05:27.518238 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:05:27.518238 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:05:27.518238 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:05:27.518238 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:05:27.518238 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:05:27.518238 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application with Lock( Aug 31 01:05:27.518238 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:05:27.518238 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:05:27.518238 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:05:27.518238 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:05:27.518238 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:05:27.518238 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:05:27.518238 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:05:27.518238 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:05:27.518238 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 01:05:27.519157 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 01:05:27.519157 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 01:05:27.519157 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Aug 31 01:05:27.521426 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 557/1089] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 01:05:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:05:27.524824 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416dc185-488b-45b4-b0f0-ff8fd65c4a0f None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:05:27.553846 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:05:27.583585 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecca49df-1f18-4df6-8067-327a5d8ad727 None None] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:27.583781 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecca49df-1f18-4df6-8067-327a5d8ad727 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:27.584067 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecca49df-1f18-4df6-8067-327a5d8ad727 None None] PATH_INFO: `/` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:27.585580 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 558/1090] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:05:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 01:05:27.590264 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb06a63b-5dad-4748-98fe-58316c25b0cd None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:27.590552 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb06a63b-5dad-4748-98fe-58316c25b0cd None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:27.590838 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb06a63b-5dad-4748-98fe-58316c25b0cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:27.609056 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-cb06a63b-5dad-4748-98fe-58316c25b0cd None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:05:27.610283 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-cb06a63b-5dad-4748-98fe-58316c25b0cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:05:27.625087 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416dc185-488b-45b4-b0f0-ff8fd65c4a0f None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:05:27.658183 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c529d25-e0fb-4c6d-b04b-58e68e1c59ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:05:27.658780 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c529d25-e0fb-4c6d-b04b-58e68e1c59ee None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:27.659059 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c529d25-e0fb-4c6d-b04b-58e68e1c59ee None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:27.659331 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c529d25-e0fb-4c6d-b04b-58e68e1c59ee None admin] PATH_INFO: `/v3/domains` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:27.659967 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c529d25-e0fb-4c6d-b04b-58e68e1c59ee None admin] RBAC: Adding query filter params (name=Default) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:05:27.660485 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c529d25-e0fb-4c6d-b04b-58e68e1c59ee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:05:27.662138 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c529d25-e0fb-4c6d-b04b-58e68e1c59ee None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:05:27.673593 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 533/1091] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 31 01:05:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:05:27.679661 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c529d25-e0fb-4c6d-b04b-58e68e1c59ee None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:05:27.682456 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 559/1092] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:05:27 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 01:05:27.762333 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:05:27.774240 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c529d25-e0fb-4c6d-b04b-58e68e1c59ee None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:05:27.807651 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b992b1-f719-4322-9bce-954cb9d8d23f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:05:27.808274 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b992b1-f719-4322-9bce-954cb9d8d23f None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:27.808560 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b992b1-f719-4322-9bce-954cb9d8d23f None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:27.808868 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b992b1-f719-4322-9bce-954cb9d8d23f None admin] PATH_INFO: `/v3/users/nova` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:27.815873 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b992b1-f719-4322-9bce-954cb9d8d23f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:05:27.817378 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b992b1-f719-4322-9bce-954cb9d8d23f None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:05:27.829109 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application [None req-48b992b1-f719-4322-9bce-954cb9d8d23f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 31 01:05:27.829109 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:05:27.829109 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:05:27.829109 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:05:27.829109 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:05:27.829109 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:05:27.829109 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:05:27.829109 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:05:27.829109 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:05:27.829109 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:05:27.829109 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:05:27.829109 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:05:27.829109 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 01:05:27.829109 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 01:05:27.829109 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 01:05:27.829109 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 01:05:27.830215 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:05:27.830215 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:05:27.830215 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 01:05:27.830215 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 01:05:27.830215 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 01:05:27.830215 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 01:05:27.830215 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:05:27.830215 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:05:27.830215 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:05:27.830215 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:05:27.830215 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:05:27.830215 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application with Lock( Aug 31 01:05:27.830215 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:05:27.830215 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:05:27.830215 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:05:27.830215 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:05:27.830215 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:05:27.830215 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:05:27.831185 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:05:27.831185 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:05:27.831185 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 01:05:27.831185 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 01:05:27.831185 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 01:05:27.831185 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 01:05:27.831185 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 01:05:27.831185 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 01:05:27.831185 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 31 01:05:27.831185 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Aug 31 01:05:27.831185 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 534/1093] 10.222.0.20 () {56 vars in 1093 bytes} [Wed Aug 31 01:05:27 2022] GET /identity/v3/users/nova => generated 82 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 01:05:27.836517 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb06a63b-5dad-4748-98fe-58316c25b0cd None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:05:27.921648 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:05:27.935882 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb06a63b-5dad-4748-98fe-58316c25b0cd None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:05:27.947400 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b992b1-f719-4322-9bce-954cb9d8d23f None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:05:27.969387 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f845b455-c15f-4d4e-982c-7f4c68e7eb25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:05:27.969990 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f845b455-c15f-4d4e-982c-7f4c68e7eb25 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:27.970269 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f845b455-c15f-4d4e-982c-7f4c68e7eb25 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:27.970542 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f845b455-c15f-4d4e-982c-7f4c68e7eb25 None admin] PATH_INFO: `/v3/users` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:27.971260 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f845b455-c15f-4d4e-982c-7f4c68e7eb25 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:05:27.971563 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f845b455-c15f-4d4e-982c-7f4c68e7eb25 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:05:27.973025 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f845b455-c15f-4d4e-982c-7f4c68e7eb25 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:05:27.991300 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 560/1094] 10.222.0.20 () {56 vars in 1133 bytes} [Wed Aug 31 01:05:27 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:05:27.996952 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f845b455-c15f-4d4e-982c-7f4c68e7eb25 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:05:28.049157 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b992b1-f719-4322-9bce-954cb9d8d23f None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:05:28.081804 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32758dc8-ae28-4135-908b-ac18bdff2eb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:05:28.082420 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32758dc8-ae28-4135-908b-ac18bdff2eb1 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:28.082698 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32758dc8-ae28-4135-908b-ac18bdff2eb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:28.082971 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32758dc8-ae28-4135-908b-ac18bdff2eb1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:28.093409 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f845b455-c15f-4d4e-982c-7f4c68e7eb25 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:05:28.093561 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32758dc8-ae28-4135-908b-ac18bdff2eb1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 01:05:28.098347 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application [None req-32758dc8-ae28-4135-908b-ac18bdff2eb1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 01:05:28.098347 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:05:28.098347 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:05:28.098347 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:05:28.098347 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:05:28.098347 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:05:28.098347 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:05:28.098347 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:05:28.098347 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:05:28.098347 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:05:28.098347 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:05:28.098347 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:05:28.098347 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 01:05:28.098347 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 01:05:28.098347 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 01:05:28.098347 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 01:05:28.099904 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 01:05:28.099904 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 01:05:28.099904 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 01:05:28.099904 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 01:05:28.099904 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:05:28.099904 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:05:28.099904 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:05:28.099904 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:05:28.099904 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:05:28.099904 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:05:28.099904 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:05:28.099904 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application with Lock( Aug 31 01:05:28.099904 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:05:28.099904 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:05:28.099904 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:05:28.099904 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:05:28.100777 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:05:28.100777 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:05:28.100777 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:05:28.100777 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:05:28.100777 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 01:05:28.100777 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 01:05:28.100777 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 01:05:28.100777 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 01:05:28.100777 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 01:05:28.100777 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 01:05:28.100777 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 01:05:28.100777 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Aug 31 01:05:28.100777 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 535/1095] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Aug 31 01:05:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 01:05:28.105767 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32758dc8-ae28-4135-908b-ac18bdff2eb1 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:05:28.116065 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:05:28.126908 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd94df71-2c63-4694-87c1-d7b18ee01c16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:05:28.127512 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd94df71-2c63-4694-87c1-d7b18ee01c16 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:28.127791 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd94df71-2c63-4694-87c1-d7b18ee01c16 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:28.128970 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd94df71-2c63-4694-87c1-d7b18ee01c16 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:28.142770 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:05:28.195021 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd94df71-2c63-4694-87c1-d7b18ee01c16 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:05:28.196454 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd94df71-2c63-4694-87c1-d7b18ee01c16 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:05:28.203232 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32758dc8-ae28-4135-908b-ac18bdff2eb1 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:05:28.236668 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365d0f86-3d9b-45e7-9dc1-492f8e5f4fa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:05:28.237286 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365d0f86-3d9b-45e7-9dc1-492f8e5f4fa8 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:28.237572 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365d0f86-3d9b-45e7-9dc1-492f8e5f4fa8 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:28.237841 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365d0f86-3d9b-45e7-9dc1-492f8e5f4fa8 None admin] PATH_INFO: `/v3/roles` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:28.238486 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365d0f86-3d9b-45e7-9dc1-492f8e5f4fa8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:05:28.238787 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365d0f86-3d9b-45e7-9dc1-492f8e5f4fa8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:05:28.240779 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365d0f86-3d9b-45e7-9dc1-492f8e5f4fa8 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:05:28.256692 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 536/1096] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Aug 31 01:05:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 01:05:28.262329 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365d0f86-3d9b-45e7-9dc1-492f8e5f4fa8 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:05:28.295352 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 561/1097] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Aug 31 01:05:27 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 31 01:05:28.300967 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd94df71-2c63-4694-87c1-d7b18ee01c16 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:05:28.330708 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:05:28.344463 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:05:28.359642 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365d0f86-3d9b-45e7-9dc1-492f8e5f4fa8 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:05:28.395065 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ba237c-2457-4d72-9e94-ea0d6984879a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:05:28.395703 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ba237c-2457-4d72-9e94-ea0d6984879a None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:28.395988 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ba237c-2457-4d72-9e94-ea0d6984879a None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:28.396264 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ba237c-2457-4d72-9e94-ea0d6984879a None admin] PATH_INFO: `/v3/users/68c5f345cd89497b905b9863997efeff` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:28.396868 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd94df71-2c63-4694-87c1-d7b18ee01c16 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:05:28.397403 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ba237c-2457-4d72-9e94-ea0d6984879a None admin] RBAC: Authorizing `identity:get_user(user_id=68c5f345cd89497b905b9863997efeff)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:05:28.398833 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ba237c-2457-4d72-9e94-ea0d6984879a None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:05:28.400396 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 537/1098] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Aug 31 01:05:28 2022] GET /identity/v3/users/68c5f345cd89497b905b9863997efeff => generated 285 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 01:05:28.406095 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ba237c-2457-4d72-9e94-ea0d6984879a None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:05:28.430977 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567bc5f5-ce7e-401e-ae2c-218ecc458a62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:05:28.431605 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567bc5f5-ce7e-401e-ae2c-218ecc458a62 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:28.431881 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567bc5f5-ce7e-401e-ae2c-218ecc458a62 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:28.432154 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567bc5f5-ce7e-401e-ae2c-218ecc458a62 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:28.442087 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567bc5f5-ce7e-401e-ae2c-218ecc458a62 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:05:28.443544 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567bc5f5-ce7e-401e-ae2c-218ecc458a62 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:05:28.458899 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application [None req-567bc5f5-ce7e-401e-ae2c-218ecc458a62 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 01:05:28.458899 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:05:28.458899 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:05:28.458899 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:05:28.458899 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:05:28.458899 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:05:28.458899 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:05:28.458899 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:05:28.458899 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:05:28.458899 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:05:28.458899 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:05:28.458899 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:05:28.458899 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 01:05:28.458899 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 01:05:28.458899 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 01:05:28.458899 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 01:05:28.459823 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:05:28.459823 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:05:28.459823 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:05:28.459823 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:05:28.459823 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:05:28.459823 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:05:28.459823 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:05:28.459823 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application with Lock( Aug 31 01:05:28.459823 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:05:28.459823 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:05:28.459823 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:05:28.459823 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:05:28.459823 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:05:28.459823 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:05:28.459823 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:05:28.459823 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:05:28.459823 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 01:05:28.461265 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 01:05:28.461265 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 01:05:28.461265 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Aug 31 01:05:28.461265 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 562/1099] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 01:05:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 01:05:28.465583 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567bc5f5-ce7e-401e-ae2c-218ecc458a62 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:05:28.502835 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ba237c-2457-4d72-9e94-ea0d6984879a None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:05:28.535975 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5311e699-edc3-49f9-a237-da9778c9b2c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:05:28.537129 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5311e699-edc3-49f9-a237-da9778c9b2c5 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:28.537410 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5311e699-edc3-49f9-a237-da9778c9b2c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:28.537681 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5311e699-edc3-49f9-a237-da9778c9b2c5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:28.545506 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:05:28.556922 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:05:28.562551 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567bc5f5-ce7e-401e-ae2c-218ecc458a62 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:05:28.595177 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0742436-aefa-49be-928c-3aedfd8ca318 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:05:28.595710 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0742436-aefa-49be-928c-3aedfd8ca318 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:28.595993 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0742436-aefa-49be-928c-3aedfd8ca318 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:28.596265 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0742436-aefa-49be-928c-3aedfd8ca318 None admin] PATH_INFO: `/v3/domains` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:28.596931 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0742436-aefa-49be-928c-3aedfd8ca318 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:05:28.597239 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0742436-aefa-49be-928c-3aedfd8ca318 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:05:28.598594 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0742436-aefa-49be-928c-3aedfd8ca318 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:05:28.605022 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5311e699-edc3-49f9-a237-da9778c9b2c5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:05:28.606921 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5311e699-edc3-49f9-a237-da9778c9b2c5 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:05:28.610912 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 563/1100] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 31 01:05:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 01:05:28.616452 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0742436-aefa-49be-928c-3aedfd8ca318 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:05:28.703718 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 538/1101] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Aug 31 01:05:28 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 31 01:05:28.709599 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5311e699-edc3-49f9-a237-da9778c9b2c5 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:05:28.713419 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0742436-aefa-49be-928c-3aedfd8ca318 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:05:28.747202 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:05:28.748466 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2868e35-65da-4a9b-85a4-e482530df40a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:05:28.749130 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2868e35-65da-4a9b-85a4-e482530df40a None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:28.749414 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2868e35-65da-4a9b-85a4-e482530df40a None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:28.749691 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2868e35-65da-4a9b-85a4-e482530df40a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:28.757054 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2868e35-65da-4a9b-85a4-e482530df40a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:05:28.769636 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2868e35-65da-4a9b-85a4-e482530df40a None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:05:28.769636 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application [None req-a2868e35-65da-4a9b-85a4-e482530df40a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 01:05:28.769636 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:05:28.769636 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:05:28.769636 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:05:28.769636 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:05:28.769636 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:05:28.769636 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:05:28.769636 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:05:28.769636 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:05:28.769636 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:05:28.769636 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:05:28.769636 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:05:28.769636 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 01:05:28.770447 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 01:05:28.770447 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 01:05:28.770447 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 01:05:28.770447 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:05:28.770447 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:05:28.770447 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:05:28.770447 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:05:28.770447 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:05:28.770447 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:05:28.770447 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:05:28.770447 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application with Lock( Aug 31 01:05:28.770447 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:05:28.770447 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:05:28.770447 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:05:28.770447 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:05:28.770447 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:05:28.770447 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:05:28.771402 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:05:28.771402 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:05:28.771402 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 01:05:28.771402 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 01:05:28.771402 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 01:05:28.771402 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 01:05:28.771402 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 01:05:28.771402 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 01:05:28.771402 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 01:05:28.771402 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Aug 31 01:05:28.771402 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 564/1102] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Aug 31 01:05:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 01:05:28.776452 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2868e35-65da-4a9b-85a4-e482530df40a None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:05:28.794322 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:05:28.805298 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5311e699-edc3-49f9-a237-da9778c9b2c5 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:05:28.839518 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dee444-bac3-4d78-aea1-22cb0fedad00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:05:28.840239 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dee444-bac3-4d78-aea1-22cb0fedad00 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:28.840589 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dee444-bac3-4d78-aea1-22cb0fedad00 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:28.840903 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dee444-bac3-4d78-aea1-22cb0fedad00 None admin] PATH_INFO: `/v3/projects/daa87f11c6484ca0bcfbf25b676324fe` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:28.842013 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6dee444-bac3-4d78-aea1-22cb0fedad00 None admin] RBAC: Authorizing `identity:get_project(project_id=daa87f11c6484ca0bcfbf25b676324fe)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:05:28.843450 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6dee444-bac3-4d78-aea1-22cb0fedad00 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:05:28.845160 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 539/1103] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Aug 31 01:05:28 2022] GET /identity/v3/projects/daa87f11c6484ca0bcfbf25b676324fe => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 01:05:28.851679 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dee444-bac3-4d78-aea1-22cb0fedad00 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:05:28.874747 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2868e35-65da-4a9b-85a4-e482530df40a None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:05:28.909915 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c9b64a-dd30-4cb4-9931-89e1ce9b3646 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:05:28.910525 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c9b64a-dd30-4cb4-9931-89e1ce9b3646 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:28.910810 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c9b64a-dd30-4cb4-9931-89e1ce9b3646 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:28.911113 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c9b64a-dd30-4cb4-9931-89e1ce9b3646 None admin] PATH_INFO: `/v3/projects` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:28.911828 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c9b64a-dd30-4cb4-9931-89e1ce9b3646 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:05:28.912139 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c9b64a-dd30-4cb4-9931-89e1ce9b3646 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:05:28.913595 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c9b64a-dd30-4cb4-9931-89e1ce9b3646 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:05:28.925197 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 565/1104] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Aug 31 01:05:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 01:05:28.931104 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c9b64a-dd30-4cb4-9931-89e1ce9b3646 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:05:28.952814 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dee444-bac3-4d78-aea1-22cb0fedad00 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:05:28.960562 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:05:28.985879 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b575963-8666-4ee5-99d6-f566776d4280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:05:28.986476 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b575963-8666-4ee5-99d6-f566776d4280 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:28.986754 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b575963-8666-4ee5-99d6-f566776d4280 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:28.987025 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b575963-8666-4ee5-99d6-f566776d4280 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:28.987739 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b575963-8666-4ee5-99d6-f566776d4280 None admin] RBAC: Adding query filter params (role.id=d009411496b8452b88e77ffb8db015a7, scope.project.id=daa87f11c6484ca0bcfbf25b676324fe, user.id=68c5f345cd89497b905b9863997efeff) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:05:28.988047 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b575963-8666-4ee5-99d6-f566776d4280 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:05:28.989547 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b575963-8666-4ee5-99d6-f566776d4280 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:05:28.995554 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 540/1105] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Aug 31 01:05:28 2022] GET /identity/v3/role_assignments?user.id=68c5f345cd89497b905b9863997efeff&scope.project.id=daa87f11c6484ca0bcfbf25b676324fe&role.id=d009411496b8452b88e77ffb8db015a7 => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 01:05:29.007900 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:05:29.033040 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c9b64a-dd30-4cb4-9931-89e1ce9b3646 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:05:29.069376 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d9d5fd-8e2d-4cd3-a06b-412f56d43dc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:05:29.070002 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d9d5fd-8e2d-4cd3-a06b-412f56d43dc6 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:29.070282 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d9d5fd-8e2d-4cd3-a06b-412f56d43dc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:29.070557 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d9d5fd-8e2d-4cd3-a06b-412f56d43dc6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:29.071297 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d9d5fd-8e2d-4cd3-a06b-412f56d43dc6 None admin] RBAC: Adding query filter params (role.id=586b679002e645439ed44fe1ca4a7cc6, scope.project.id=60b2abc15c2849ba8a1ed92117cc981b, user.id=8744c9de803048f583d0317d01f6a540) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:05:29.071608 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d9d5fd-8e2d-4cd3-a06b-412f56d43dc6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:05:29.073109 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d9d5fd-8e2d-4cd3-a06b-412f56d43dc6 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:05:29.079705 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 566/1106] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Aug 31 01:05:28 2022] GET /identity/v3/role_assignments?user.id=8744c9de803048f583d0317d01f6a540&scope.project.id=60b2abc15c2849ba8a1ed92117cc981b&role.id=586b679002e645439ed44fe1ca4a7cc6 => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:05:30.931786 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b6e030-2f26-4462-b325-a88d412a9fac None None] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:30.932253 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b6e030-2f26-4462-b325-a88d412a9fac None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:30.932348 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b6e030-2f26-4462-b325-a88d412a9fac None None] PATH_INFO: `/` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:30.933700 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 541/1107] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:05:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:05:30.938728 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5fae98-31ab-45da-840b-1b2e2267e867 None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:30.939136 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5fae98-31ab-45da-840b-1b2e2267e867 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:30.939294 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5fae98-31ab-45da-840b-1b2e2267e867 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:30.957287 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-8b5fae98-31ab-45da-840b-1b2e2267e867 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:05:30.958995 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-8b5fae98-31ab-45da-840b-1b2e2267e867 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:05:31.026919 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 567/1108] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:05:30 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 01:05:31.294845 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b6e030-2f26-4462-b325-a88d412a9fac None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:05:31.395827 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b6e030-2f26-4462-b325-a88d412a9fac None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:05:31.429305 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a639e2b8-45ee-49db-8420-9ffd97c6bf78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:05:31.429925 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a639e2b8-45ee-49db-8420-9ffd97c6bf78 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:31.430206 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a639e2b8-45ee-49db-8420-9ffd97c6bf78 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:31.430484 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a639e2b8-45ee-49db-8420-9ffd97c6bf78 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:31.431588 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a639e2b8-45ee-49db-8420-9ffd97c6bf78 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:05:31.433066 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a639e2b8-45ee-49db-8420-9ffd97c6bf78 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:05:31.434623 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 542/1109] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 01:05:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:05:31.440272 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5fae98-31ab-45da-840b-1b2e2267e867 None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:05:31.507039 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:05:31.537526 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5fae98-31ab-45da-840b-1b2e2267e867 None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:05:31.569883 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc08380-30e9-415a-b19f-593b57d8942a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:05:31.570423 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc08380-30e9-415a-b19f-593b57d8942a None admin] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:31.570706 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc08380-30e9-415a-b19f-593b57d8942a None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:31.570983 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc08380-30e9-415a-b19f-593b57d8942a None admin] PATH_INFO: `/v3/users` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:31.571640 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc08380-30e9-415a-b19f-593b57d8942a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:05:31.573097 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc08380-30e9-415a-b19f-593b57d8942a None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:05:31.650988 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 568/1110] 10.222.0.20 () {60 vars in 1132 bytes} [Wed Aug 31 01:05:31 2022] POST /identity/v3/users => generated 286 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:05:33.598251 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4369ae2-43f9-409c-a0a8-e9b1c3056914 None None] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:33.599323 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4369ae2-43f9-409c-a0a8-e9b1c3056914 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:33.599323 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4369ae2-43f9-409c-a0a8-e9b1c3056914 None None] PATH_INFO: `/` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:33.600657 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 543/1111] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:05:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:05:33.605701 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9215398-490e-4e32-b4fe-312f5ca21de0 None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:33.606089 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9215398-490e-4e32-b4fe-312f5ca21de0 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:33.606265 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9215398-490e-4e32-b4fe-312f5ca21de0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:33.625537 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-a9215398-490e-4e32-b4fe-312f5ca21de0 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:05:33.626747 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-a9215398-490e-4e32-b4fe-312f5ca21de0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:05:33.690479 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 569/1112] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:05:33 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 01:05:33.959373 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4369ae2-43f9-409c-a0a8-e9b1c3056914 None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:05:34.000456 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:05:34.058791 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4369ae2-43f9-409c-a0a8-e9b1c3056914 None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:05:34.091782 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe29e58-2d1f-4b85-945b-4fcb98638e9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:05:34.092398 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe29e58-2d1f-4b85-945b-4fcb98638e9c None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:34.092678 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe29e58-2d1f-4b85-945b-4fcb98638e9c None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:34.092986 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe29e58-2d1f-4b85-945b-4fcb98638e9c None admin] PATH_INFO: `/v3/roles/bebc77e8dfd54dbba7079f93218e7c7d` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:34.094353 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe29e58-2d1f-4b85-945b-4fcb98638e9c None admin] RBAC: Authorizing `identity:get_role(role_id=bebc77e8dfd54dbba7079f93218e7c7d)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:05:34.095725 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe29e58-2d1f-4b85-945b-4fcb98638e9c None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:05:34.097062 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 544/1113] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Aug 31 01:05:33 2022] GET /identity/v3/roles/bebc77e8dfd54dbba7079f93218e7c7d => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:05:34.103241 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9215398-490e-4e32-b4fe-312f5ca21de0 None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:05:34.155979 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:05:34.197682 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9215398-490e-4e32-b4fe-312f5ca21de0 None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:05:34.230713 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b7550a-374d-47d9-be0e-e7313d0a0b93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:05:34.231329 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b7550a-374d-47d9-be0e-e7313d0a0b93 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:34.231614 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b7550a-374d-47d9-be0e-e7313d0a0b93 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:34.231890 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b7550a-374d-47d9-be0e-e7313d0a0b93 None admin] PATH_INFO: `/v3/users/191bff24a6df46fa8c2350117b82c97d` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:34.244872 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b7550a-374d-47d9-be0e-e7313d0a0b93 None admin] RBAC: Authorizing `identity:get_user(user_id=191bff24a6df46fa8c2350117b82c97d)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:05:34.246311 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b7550a-374d-47d9-be0e-e7313d0a0b93 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:05:34.247880 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 570/1114] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Aug 31 01:05:34 2022] GET /identity/v3/users/191bff24a6df46fa8c2350117b82c97d => generated 286 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:05:34.253447 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe29e58-2d1f-4b85-945b-4fcb98638e9c None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:05:34.350223 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe29e58-2d1f-4b85-945b-4fcb98638e9c None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:05:34.357299 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:05:34.382982 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bca90d0-b020-4185-b2a7-747921284dd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:05:34.383579 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bca90d0-b020-4185-b2a7-747921284dd2 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:34.383862 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bca90d0-b020-4185-b2a7-747921284dd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:34.384137 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bca90d0-b020-4185-b2a7-747921284dd2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:34.449034 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bca90d0-b020-4185-b2a7-747921284dd2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:05:34.450729 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bca90d0-b020-4185-b2a7-747921284dd2 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:05:34.545912 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 545/1115] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Aug 31 01:05:34 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 01:05:34.552339 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b7550a-374d-47d9-be0e-e7313d0a0b93 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:05:34.646766 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b7550a-374d-47d9-be0e-e7313d0a0b93 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:05:34.659928 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:05:34.679393 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f8e1eb-cca2-4874-80ce-7f60f61130dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:05:34.680006 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f8e1eb-cca2-4874-80ce-7f60f61130dc None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:34.680289 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f8e1eb-cca2-4874-80ce-7f60f61130dc None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:34.680563 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f8e1eb-cca2-4874-80ce-7f60f61130dc None admin] PATH_INFO: `/v3/projects/daa87f11c6484ca0bcfbf25b676324fe` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:34.681683 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f8e1eb-cca2-4874-80ce-7f60f61130dc None admin] RBAC: Authorizing `identity:get_project(project_id=daa87f11c6484ca0bcfbf25b676324fe)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:05:34.683099 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f8e1eb-cca2-4874-80ce-7f60f61130dc None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:05:34.684786 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 571/1116] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Aug 31 01:05:34 2022] GET /identity/v3/projects/daa87f11c6484ca0bcfbf25b676324fe => generated 311 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:05:34.690666 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bca90d0-b020-4185-b2a7-747921284dd2 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:05:34.699108 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:05:34.786152 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bca90d0-b020-4185-b2a7-747921284dd2 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:05:34.819302 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c4b3a1-38c7-41c2-af01-d938ef435059 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:05:34.819889 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c4b3a1-38c7-41c2-af01-d938ef435059 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:34.820171 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c4b3a1-38c7-41c2-af01-d938ef435059 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:34.820446 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c4b3a1-38c7-41c2-af01-d938ef435059 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:34.821196 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c4b3a1-38c7-41c2-af01-d938ef435059 None admin] RBAC: Adding query filter params (role.id=bebc77e8dfd54dbba7079f93218e7c7d, scope.project.id=daa87f11c6484ca0bcfbf25b676324fe, user.id=191bff24a6df46fa8c2350117b82c97d) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:05:34.821505 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c4b3a1-38c7-41c2-af01-d938ef435059 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:05:34.822933 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c4b3a1-38c7-41c2-af01-d938ef435059 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:05:34.828365 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 546/1117] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Aug 31 01:05:34 2022] GET /identity/v3/role_assignments?user.id=191bff24a6df46fa8c2350117b82c97d&scope.project.id=daa87f11c6484ca0bcfbf25b676324fe&role.id=bebc77e8dfd54dbba7079f93218e7c7d => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:05:36.679209 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eee4f3f-eaf4-4b4d-bb33-229e1079aaf7 None None] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:36.679761 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eee4f3f-eaf4-4b4d-bb33-229e1079aaf7 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:36.679761 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eee4f3f-eaf4-4b4d-bb33-229e1079aaf7 None None] PATH_INFO: `/` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:36.681128 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 572/1118] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:05:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:05:36.686264 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03372f8d-a289-4cc3-8709-efd643483078 None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:36.686647 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03372f8d-a289-4cc3-8709-efd643483078 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:36.686822 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03372f8d-a289-4cc3-8709-efd643483078 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:36.704836 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-03372f8d-a289-4cc3-8709-efd643483078 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:05:36.709185 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-03372f8d-a289-4cc3-8709-efd643483078 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:05:36.766467 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:05:36.771297 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 547/1119] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:05:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 01:05:37.035890 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eee4f3f-eaf4-4b4d-bb33-229e1079aaf7 None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:05:37.132911 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eee4f3f-eaf4-4b4d-bb33-229e1079aaf7 None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:05:37.165346 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebc1317-c26b-4a04-a441-bcd43202c0a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:05:37.165952 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebc1317-c26b-4a04-a441-bcd43202c0a6 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:37.166231 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebc1317-c26b-4a04-a441-bcd43202c0a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:37.166499 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebc1317-c26b-4a04-a441-bcd43202c0a6 None admin] PATH_INFO: `/v3/roles/bebc77e8dfd54dbba7079f93218e7c7d` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:37.167863 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebc1317-c26b-4a04-a441-bcd43202c0a6 None admin] RBAC: Authorizing `identity:get_role(role_id=bebc77e8dfd54dbba7079f93218e7c7d)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:05:37.169249 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebc1317-c26b-4a04-a441-bcd43202c0a6 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:05:37.170517 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 573/1120] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Aug 31 01:05:37 2022] GET /identity/v3/roles/bebc77e8dfd54dbba7079f93218e7c7d => generated 223 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:05:37.176346 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03372f8d-a289-4cc3-8709-efd643483078 None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:05:37.272336 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03372f8d-a289-4cc3-8709-efd643483078 None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:05:37.304657 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c4ae68-c092-4361-a643-c681aaa53b01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:05:37.305236 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c4ae68-c092-4361-a643-c681aaa53b01 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:37.305514 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c4ae68-c092-4361-a643-c681aaa53b01 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:37.305788 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c4ae68-c092-4361-a643-c681aaa53b01 None admin] PATH_INFO: `/v3/users/191bff24a6df46fa8c2350117b82c97d` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:37.306884 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c4ae68-c092-4361-a643-c681aaa53b01 None admin] RBAC: Authorizing `identity:get_user(user_id=191bff24a6df46fa8c2350117b82c97d)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:05:37.308305 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c4ae68-c092-4361-a643-c681aaa53b01 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:05:37.309867 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 548/1121] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Aug 31 01:05:37 2022] GET /identity/v3/users/191bff24a6df46fa8c2350117b82c97d => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:05:37.315365 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebc1317-c26b-4a04-a441-bcd43202c0a6 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:05:37.365309 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:05:37.412589 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebc1317-c26b-4a04-a441-bcd43202c0a6 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:05:37.445641 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e4121d-20e1-4795-87b1-9244e96ecd3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:05:37.446259 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e4121d-20e1-4795-87b1-9244e96ecd3d None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:37.446541 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e4121d-20e1-4795-87b1-9244e96ecd3d None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:37.446816 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e4121d-20e1-4795-87b1-9244e96ecd3d None admin] PATH_INFO: `/v3/projects/daa87f11c6484ca0bcfbf25b676324fe` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:37.447919 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e4121d-20e1-4795-87b1-9244e96ecd3d None admin] RBAC: Authorizing `identity:get_project(project_id=daa87f11c6484ca0bcfbf25b676324fe)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:05:37.449378 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e4121d-20e1-4795-87b1-9244e96ecd3d None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:05:37.451248 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 574/1122] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Aug 31 01:05:37 2022] GET /identity/v3/projects/daa87f11c6484ca0bcfbf25b676324fe => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:05:37.457070 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c4ae68-c092-4361-a643-c681aaa53b01 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:05:37.521172 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:05:37.553395 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c4ae68-c092-4361-a643-c681aaa53b01 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:05:37.586201 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3283e1cb-e59d-45d8-8ceb-b14609357c54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:05:37.586781 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3283e1cb-e59d-45d8-8ceb-b14609357c54 None admin] REQUEST_METHOD: `PUT` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:37.587061 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3283e1cb-e59d-45d8-8ceb-b14609357c54 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:37.587337 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3283e1cb-e59d-45d8-8ceb-b14609357c54 None admin] PATH_INFO: `/v3/projects/daa87f11c6484ca0bcfbf25b676324fe/users/191bff24a6df46fa8c2350117b82c97d/roles/bebc77e8dfd54dbba7079f93218e7c7d` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:37.589446 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3283e1cb-e59d-45d8-8ceb-b14609357c54 None admin] RBAC: Authorizing `identity:create_grant(project_id=daa87f11c6484ca0bcfbf25b676324fe, user_id=191bff24a6df46fa8c2350117b82c97d, role_id=bebc77e8dfd54dbba7079f93218e7c7d)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:05:37.590861 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3283e1cb-e59d-45d8-8ceb-b14609357c54 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:05:37.644750 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 549/1123] 10.222.0.20 () {58 vars in 1439 bytes} [Wed Aug 31 01:05:37 2022] PUT /identity/v3/projects/daa87f11c6484ca0bcfbf25b676324fe/users/191bff24a6df46fa8c2350117b82c97d/roles/bebc77e8dfd54dbba7079f93218e7c7d => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:05:39.486487 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a5c32a-551d-44c7-80f1-e0a25f0b5396 None None] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:39.487042 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a5c32a-551d-44c7-80f1-e0a25f0b5396 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:39.487042 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a5c32a-551d-44c7-80f1-e0a25f0b5396 None None] PATH_INFO: `/` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:39.488377 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 575/1124] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:05:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:05:39.493384 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d95de6a-c5cc-48f5-b854-d8298880bc14 None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:39.493801 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d95de6a-c5cc-48f5-b854-d8298880bc14 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:39.493942 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d95de6a-c5cc-48f5-b854-d8298880bc14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:39.512145 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-7d95de6a-c5cc-48f5-b854-d8298880bc14 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:05:39.513352 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-7d95de6a-c5cc-48f5-b854-d8298880bc14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:05:39.577752 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 550/1125] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:05:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 01:05:39.844864 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a5c32a-551d-44c7-80f1-e0a25f0b5396 None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:05:39.943948 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a5c32a-551d-44c7-80f1-e0a25f0b5396 None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:05:39.961295 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:05:39.977260 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8e6946-bf67-4d40-b066-efa8df94e823 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:05:39.978347 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8e6946-bf67-4d40-b066-efa8df94e823 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:39.978347 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8e6946-bf67-4d40-b066-efa8df94e823 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:39.978536 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8e6946-bf67-4d40-b066-efa8df94e823 None admin] PATH_INFO: `/v3/roles/bebc77e8dfd54dbba7079f93218e7c7d` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:39.979815 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8e6946-bf67-4d40-b066-efa8df94e823 None admin] RBAC: Authorizing `identity:get_role(role_id=bebc77e8dfd54dbba7079f93218e7c7d)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:05:39.981257 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8e6946-bf67-4d40-b066-efa8df94e823 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:05:39.982561 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 576/1126] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Aug 31 01:05:39 2022] GET /identity/v3/roles/bebc77e8dfd54dbba7079f93218e7c7d => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:05:39.988502 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d95de6a-c5cc-48f5-b854-d8298880bc14 None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:05:40.044176 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:05:40.087830 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d95de6a-c5cc-48f5-b854-d8298880bc14 None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:05:40.121670 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e227322-4bc3-4794-b5c7-4871bad8237e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:05:40.122234 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e227322-4bc3-4794-b5c7-4871bad8237e None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:40.122516 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e227322-4bc3-4794-b5c7-4871bad8237e None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:40.122790 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e227322-4bc3-4794-b5c7-4871bad8237e None admin] PATH_INFO: `/v3/users/191bff24a6df46fa8c2350117b82c97d` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:40.123897 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e227322-4bc3-4794-b5c7-4871bad8237e None admin] RBAC: Authorizing `identity:get_user(user_id=191bff24a6df46fa8c2350117b82c97d)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:05:40.125371 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e227322-4bc3-4794-b5c7-4871bad8237e None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:05:40.126929 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 551/1127] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Aug 31 01:05:39 2022] GET /identity/v3/users/191bff24a6df46fa8c2350117b82c97d => generated 286 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:05:40.132774 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8e6946-bf67-4d40-b066-efa8df94e823 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:05:40.228423 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8e6946-bf67-4d40-b066-efa8df94e823 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:05:40.260837 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba8bd1f-e867-4bb9-bac8-2de8a0b02aac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:05:40.261431 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba8bd1f-e867-4bb9-bac8-2de8a0b02aac None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:40.261712 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba8bd1f-e867-4bb9-bac8-2de8a0b02aac None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:40.261986 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba8bd1f-e867-4bb9-bac8-2de8a0b02aac None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:40.313690 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:05:40.326721 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba8bd1f-e867-4bb9-bac8-2de8a0b02aac None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:05:40.328163 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba8bd1f-e867-4bb9-bac8-2de8a0b02aac None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:05:40.421990 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 577/1128] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Aug 31 01:05:40 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 01:05:40.428489 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e227322-4bc3-4794-b5c7-4871bad8237e None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:05:40.524266 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e227322-4bc3-4794-b5c7-4871bad8237e None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:05:40.548811 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:05:40.557260 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70b312a-b364-4956-8aae-138fe8768973 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:05:40.557939 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70b312a-b364-4956-8aae-138fe8768973 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:40.558132 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70b312a-b364-4956-8aae-138fe8768973 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:40.558405 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70b312a-b364-4956-8aae-138fe8768973 None admin] PATH_INFO: `/v3/projects/daa87f11c6484ca0bcfbf25b676324fe` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:40.559493 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70b312a-b364-4956-8aae-138fe8768973 None admin] RBAC: Authorizing `identity:get_project(project_id=daa87f11c6484ca0bcfbf25b676324fe)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:05:40.560950 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70b312a-b364-4956-8aae-138fe8768973 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:05:40.562537 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 552/1129] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Aug 31 01:05:40 2022] GET /identity/v3/projects/daa87f11c6484ca0bcfbf25b676324fe => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:05:40.568624 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba8bd1f-e867-4bb9-bac8-2de8a0b02aac None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:05:40.649391 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:05:40.664122 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba8bd1f-e867-4bb9-bac8-2de8a0b02aac None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:05:40.696467 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937f3454-ad41-437f-8e77-e61cd3654c17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:05:40.697031 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937f3454-ad41-437f-8e77-e61cd3654c17 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:40.697315 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937f3454-ad41-437f-8e77-e61cd3654c17 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:40.697589 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937f3454-ad41-437f-8e77-e61cd3654c17 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:40.698305 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937f3454-ad41-437f-8e77-e61cd3654c17 None admin] RBAC: Adding query filter params (role.id=bebc77e8dfd54dbba7079f93218e7c7d, scope.project.id=daa87f11c6484ca0bcfbf25b676324fe, user.id=191bff24a6df46fa8c2350117b82c97d) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:05:40.698615 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937f3454-ad41-437f-8e77-e61cd3654c17 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:05:40.700300 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937f3454-ad41-437f-8e77-e61cd3654c17 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:05:40.705828 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 578/1130] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Aug 31 01:05:40 2022] GET /identity/v3/role_assignments?user.id=191bff24a6df46fa8c2350117b82c97d&scope.project.id=daa87f11c6484ca0bcfbf25b676324fe&role.id=bebc77e8dfd54dbba7079f93218e7c7d => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:05:42.592712 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05ff214-8788-4635-a871-ee62ed0061cb None None] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:42.593388 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05ff214-8788-4635-a871-ee62ed0061cb None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:42.593388 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05ff214-8788-4635-a871-ee62ed0061cb None None] PATH_INFO: `/` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:42.594634 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 553/1131] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:05:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:05:42.599703 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54380aeb-86d7-42dc-a114-b093fa368c20 None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:42.600097 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54380aeb-86d7-42dc-a114-b093fa368c20 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:42.600257 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54380aeb-86d7-42dc-a114-b093fa368c20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:42.617929 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-54380aeb-86d7-42dc-a114-b093fa368c20 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:05:42.619106 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-54380aeb-86d7-42dc-a114-b093fa368c20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:05:42.683081 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 579/1132] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:05:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 01:05:42.950104 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05ff214-8788-4635-a871-ee62ed0061cb None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:05:43.046905 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05ff214-8788-4635-a871-ee62ed0061cb None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:05:43.079850 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98faec91-36b1-496a-8acf-e78a702df4a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:05:43.080460 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98faec91-36b1-496a-8acf-e78a702df4a1 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:43.080763 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98faec91-36b1-496a-8acf-e78a702df4a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:43.081039 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98faec91-36b1-496a-8acf-e78a702df4a1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:43.082128 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98faec91-36b1-496a-8acf-e78a702df4a1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:05:43.083583 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98faec91-36b1-496a-8acf-e78a702df4a1 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:05:43.085154 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 554/1133] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 01:05:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 01:05:43.090867 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54380aeb-86d7-42dc-a114-b093fa368c20 None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:05:43.167413 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:05:43.186152 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54380aeb-86d7-42dc-a114-b093fa368c20 None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:05:43.218593 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd73c25-6fbb-4fdd-8f3d-47a1d115fdb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:05:43.219189 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd73c25-6fbb-4fdd-8f3d-47a1d115fdb1 None admin] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:43.219471 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd73c25-6fbb-4fdd-8f3d-47a1d115fdb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:43.219747 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd73c25-6fbb-4fdd-8f3d-47a1d115fdb1 None admin] PATH_INFO: `/v3/projects` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:43.220392 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd73c25-6fbb-4fdd-8f3d-47a1d115fdb1 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:05:43.221846 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd73c25-6fbb-4fdd-8f3d-47a1d115fdb1 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:05:43.320214 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 580/1134] 10.222.0.20 () {60 vars in 1141 bytes} [Wed Aug 31 01:05:43 2022] POST /identity/v3/projects => generated 311 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:05:45.216075 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dde70c-781d-4b74-8a76-9321d34dd68f None None] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:45.216608 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dde70c-781d-4b74-8a76-9321d34dd68f None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:45.216608 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dde70c-781d-4b74-8a76-9321d34dd68f None None] PATH_INFO: `/` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:45.217996 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 555/1135] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:05:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:05:45.223015 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e699b6-6cbc-47fd-bfee-296d0a4cb8b2 None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:45.223479 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e699b6-6cbc-47fd-bfee-296d0a4cb8b2 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:45.223626 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e699b6-6cbc-47fd-bfee-296d0a4cb8b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:45.242509 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-26e699b6-6cbc-47fd-bfee-296d0a4cb8b2 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:05:45.243702 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-26e699b6-6cbc-47fd-bfee-296d0a4cb8b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:05:45.306532 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 581/1136] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:05:45 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 01:05:45.576306 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dde70c-781d-4b74-8a76-9321d34dd68f None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:05:45.637300 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:05:45.674958 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dde70c-781d-4b74-8a76-9321d34dd68f None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:05:45.708946 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3ff962-0fc4-48a8-9242-a3c3e3e5e423 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:05:45.709551 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3ff962-0fc4-48a8-9242-a3c3e3e5e423 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:45.709825 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3ff962-0fc4-48a8-9242-a3c3e3e5e423 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:45.710097 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3ff962-0fc4-48a8-9242-a3c3e3e5e423 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:45.711185 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3ff962-0fc4-48a8-9242-a3c3e3e5e423 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:05:45.712607 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3ff962-0fc4-48a8-9242-a3c3e3e5e423 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:05:45.714167 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 556/1137] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 01:05:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:05:45.719837 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e699b6-6cbc-47fd-bfee-296d0a4cb8b2 None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:05:45.778542 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:05:45.815724 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e699b6-6cbc-47fd-bfee-296d0a4cb8b2 None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:05:45.849543 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfd112f-77a9-4fca-81ea-96855b162502 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:05:45.850104 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfd112f-77a9-4fca-81ea-96855b162502 None admin] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:45.850387 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfd112f-77a9-4fca-81ea-96855b162502 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:45.850661 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfd112f-77a9-4fca-81ea-96855b162502 None admin] PATH_INFO: `/v3/users` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:45.851302 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfd112f-77a9-4fca-81ea-96855b162502 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:05:45.852746 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfd112f-77a9-4fca-81ea-96855b162502 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:05:45.953046 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 582/1138] 10.222.0.20 () {60 vars in 1132 bytes} [Wed Aug 31 01:05:45 2022] POST /identity/v3/users => generated 286 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:05:47.888400 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664a8452-0a82-483f-a8a8-930b91144f5f None None] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:47.888991 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664a8452-0a82-483f-a8a8-930b91144f5f None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:47.888991 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664a8452-0a82-483f-a8a8-930b91144f5f None None] PATH_INFO: `/` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:47.890363 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 557/1139] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:05:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:05:47.895690 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192a3632-6cc6-4b0a-83b6-f8b5b9a67d51 None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:47.896643 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192a3632-6cc6-4b0a-83b6-f8b5b9a67d51 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:47.897243 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192a3632-6cc6-4b0a-83b6-f8b5b9a67d51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:47.914462 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-192a3632-6cc6-4b0a-83b6-f8b5b9a67d51 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:05:47.915648 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-192a3632-6cc6-4b0a-83b6-f8b5b9a67d51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:05:47.980195 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 583/1140] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:05:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 01:05:48.247550 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664a8452-0a82-483f-a8a8-930b91144f5f None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:05:48.344345 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664a8452-0a82-483f-a8a8-930b91144f5f None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:05:48.372120 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:05:48.377582 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6838e65-e5a0-47b1-b90a-aa952d3c8b1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:05:48.378198 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6838e65-e5a0-47b1-b90a-aa952d3c8b1e None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:48.378473 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6838e65-e5a0-47b1-b90a-aa952d3c8b1e None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:48.378744 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6838e65-e5a0-47b1-b90a-aa952d3c8b1e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:48.388223 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6838e65-e5a0-47b1-b90a-aa952d3c8b1e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 01:05:48.393025 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application [None req-b6838e65-e5a0-47b1-b90a-aa952d3c8b1e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 01:05:48.393025 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:05:48.393025 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:05:48.393025 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:05:48.393025 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:05:48.393025 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:05:48.393025 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:05:48.393025 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:05:48.393025 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:05:48.393025 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:05:48.393025 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:05:48.393025 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:05:48.393025 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 01:05:48.393025 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 01:05:48.393025 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 01:05:48.393025 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 01:05:48.403725 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 01:05:48.403725 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 01:05:48.403725 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 01:05:48.403725 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 01:05:48.403725 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:05:48.403725 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:05:48.403725 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:05:48.403725 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:05:48.403725 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:05:48.403725 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:05:48.403725 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:05:48.403725 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application with Lock( Aug 31 01:05:48.403725 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:05:48.403725 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:05:48.403725 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:05:48.403725 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:05:48.404862 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:05:48.404862 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:05:48.404862 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:05:48.404862 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:05:48.404862 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 01:05:48.404862 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 01:05:48.404862 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 01:05:48.404862 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 01:05:48.404862 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 01:05:48.404862 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 01:05:48.404862 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 01:05:48.404862 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Aug 31 01:05:48.404862 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 558/1141] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Aug 31 01:05:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:05:48.405777 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192a3632-6cc6-4b0a-83b6-f8b5b9a67d51 None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:05:48.447695 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:05:48.496956 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192a3632-6cc6-4b0a-83b6-f8b5b9a67d51 None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:05:48.529618 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf16d76-f80e-4548-aa4c-92487a20f146 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:05:48.530156 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf16d76-f80e-4548-aa4c-92487a20f146 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:48.530440 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf16d76-f80e-4548-aa4c-92487a20f146 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:48.530718 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf16d76-f80e-4548-aa4c-92487a20f146 None admin] PATH_INFO: `/v3/roles` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:48.531376 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf16d76-f80e-4548-aa4c-92487a20f146 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:05:48.531682 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf16d76-f80e-4548-aa4c-92487a20f146 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:05:48.533074 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf16d76-f80e-4548-aa4c-92487a20f146 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:05:48.541590 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 584/1142] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Aug 31 01:05:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:05:48.547330 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6838e65-e5a0-47b1-b90a-aa952d3c8b1e None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:05:48.643300 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6838e65-e5a0-47b1-b90a-aa952d3c8b1e None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:05:48.675770 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5ce2b7-7364-436f-b991-e2a45228695e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:05:48.677085 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5ce2b7-7364-436f-b991-e2a45228695e None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:48.677362 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5ce2b7-7364-436f-b991-e2a45228695e None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:48.677631 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5ce2b7-7364-436f-b991-e2a45228695e None admin] PATH_INFO: `/v3/users/59ed194b0ff2451fb7b1f6c60e5e0801` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:48.690170 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5ce2b7-7364-436f-b991-e2a45228695e None admin] RBAC: Authorizing `identity:get_user(user_id=59ed194b0ff2451fb7b1f6c60e5e0801)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:05:48.691600 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5ce2b7-7364-436f-b991-e2a45228695e None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:05:48.693228 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 559/1143] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Aug 31 01:05:48 2022] GET /identity/v3/users/59ed194b0ff2451fb7b1f6c60e5e0801 => generated 286 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:05:48.698998 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf16d76-f80e-4548-aa4c-92487a20f146 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:05:48.795677 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf16d76-f80e-4548-aa4c-92487a20f146 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:05:48.805400 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:05:48.827694 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4200c743-41b7-4173-a4da-fd8337d348e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:05:48.828288 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4200c743-41b7-4173-a4da-fd8337d348e2 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:48.828571 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4200c743-41b7-4173-a4da-fd8337d348e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:48.828878 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4200c743-41b7-4173-a4da-fd8337d348e2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:48.895646 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4200c743-41b7-4173-a4da-fd8337d348e2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:05:48.897130 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4200c743-41b7-4173-a4da-fd8337d348e2 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:05:48.992459 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 585/1144] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Aug 31 01:05:48 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 01:05:48.998331 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5ce2b7-7364-436f-b991-e2a45228695e None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:05:49.046315 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:05:49.094852 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5ce2b7-7364-436f-b991-e2a45228695e None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:05:49.127636 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4177f47d-3610-425d-a66d-5ab668769007 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:05:49.128255 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4177f47d-3610-425d-a66d-5ab668769007 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:49.128540 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4177f47d-3610-425d-a66d-5ab668769007 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:49.128845 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4177f47d-3610-425d-a66d-5ab668769007 None admin] PATH_INFO: `/v3/projects/d70a08d6795b4b3fa382f511754d30cb` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:49.130500 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4177f47d-3610-425d-a66d-5ab668769007 None admin] RBAC: Authorizing `identity:get_project(project_id=d70a08d6795b4b3fa382f511754d30cb)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:05:49.131937 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4177f47d-3610-425d-a66d-5ab668769007 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:05:49.133578 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 560/1145] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Aug 31 01:05:48 2022] GET /identity/v3/projects/d70a08d6795b4b3fa382f511754d30cb => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:05:49.139565 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4200c743-41b7-4173-a4da-fd8337d348e2 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:05:49.151345 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:05:49.235771 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4200c743-41b7-4173-a4da-fd8337d348e2 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:05:49.267736 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0362c78-b6e1-44c4-9369-99bc08b79433 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:05:49.268322 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0362c78-b6e1-44c4-9369-99bc08b79433 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:49.268596 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0362c78-b6e1-44c4-9369-99bc08b79433 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:49.268895 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0362c78-b6e1-44c4-9369-99bc08b79433 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:49.269600 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0362c78-b6e1-44c4-9369-99bc08b79433 None admin] RBAC: Adding query filter params (role.id=d009411496b8452b88e77ffb8db015a7, scope.project.id=d70a08d6795b4b3fa382f511754d30cb, user.id=59ed194b0ff2451fb7b1f6c60e5e0801) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:05:49.270525 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0362c78-b6e1-44c4-9369-99bc08b79433 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:05:49.271929 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0362c78-b6e1-44c4-9369-99bc08b79433 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:05:49.277037 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 586/1146] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Aug 31 01:05:49 2022] GET /identity/v3/role_assignments?user.id=59ed194b0ff2451fb7b1f6c60e5e0801&scope.project.id=d70a08d6795b4b3fa382f511754d30cb&role.id=d009411496b8452b88e77ffb8db015a7 => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:05:51.138109 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa3a03e-3e3d-47b5-8a1d-7628d5af4681 None None] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:51.138641 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa3a03e-3e3d-47b5-8a1d-7628d5af4681 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:51.138641 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa3a03e-3e3d-47b5-8a1d-7628d5af4681 None None] PATH_INFO: `/` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:51.140003 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 561/1147] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:05:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:05:51.145233 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52acf1d-5015-4abd-b0a3-24b185927c68 None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:51.145650 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52acf1d-5015-4abd-b0a3-24b185927c68 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:51.145785 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52acf1d-5015-4abd-b0a3-24b185927c68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:51.164283 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-d52acf1d-5015-4abd-b0a3-24b185927c68 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:05:51.165428 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-d52acf1d-5015-4abd-b0a3-24b185927c68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:05:51.234113 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 587/1148] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:05:51 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 01:05:51.499981 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa3a03e-3e3d-47b5-8a1d-7628d5af4681 None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:05:51.598909 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa3a03e-3e3d-47b5-8a1d-7628d5af4681 None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:05:51.615029 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:05:51.631499 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcde2683-d1d6-450c-a229-70870b9d2e44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:05:51.632105 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcde2683-d1d6-450c-a229-70870b9d2e44 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:51.632383 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcde2683-d1d6-450c-a229-70870b9d2e44 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:51.636236 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcde2683-d1d6-450c-a229-70870b9d2e44 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:51.642171 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcde2683-d1d6-450c-a229-70870b9d2e44 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 01:05:51.646937 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application [None req-bcde2683-d1d6-450c-a229-70870b9d2e44 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 01:05:51.646937 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:05:51.646937 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:05:51.646937 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:05:51.646937 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:05:51.646937 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:05:51.646937 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:05:51.646937 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:05:51.646937 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:05:51.646937 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:05:51.646937 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:05:51.646937 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:05:51.646937 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 01:05:51.646937 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 01:05:51.646937 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 01:05:51.646937 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 01:05:51.648197 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 01:05:51.648197 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 01:05:51.648197 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 01:05:51.648197 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 01:05:51.648197 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:05:51.648197 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:05:51.648197 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:05:51.648197 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:05:51.648197 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:05:51.648197 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:05:51.648197 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:05:51.648197 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application with Lock( Aug 31 01:05:51.648197 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:05:51.648197 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:05:51.648197 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:05:51.648197 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:05:51.649394 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:05:51.649394 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:05:51.649394 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:05:51.649394 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:05:51.649394 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 01:05:51.649394 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 01:05:51.649394 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 01:05:51.649394 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 01:05:51.649394 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 01:05:51.649394 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 01:05:51.649394 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 01:05:51.649394 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Aug 31 01:05:51.649394 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 562/1149] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Aug 31 01:05:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:05:51.653972 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52acf1d-5015-4abd-b0a3-24b185927c68 None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:05:51.656988 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:05:51.750153 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52acf1d-5015-4abd-b0a3-24b185927c68 None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:05:51.783024 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c75522f-575b-4965-8bec-597bbbc7b4d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:05:51.783618 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c75522f-575b-4965-8bec-597bbbc7b4d7 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:51.783899 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c75522f-575b-4965-8bec-597bbbc7b4d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:51.784172 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c75522f-575b-4965-8bec-597bbbc7b4d7 None admin] PATH_INFO: `/v3/roles` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:51.784852 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c75522f-575b-4965-8bec-597bbbc7b4d7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:05:51.785155 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c75522f-575b-4965-8bec-597bbbc7b4d7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:05:51.787064 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c75522f-575b-4965-8bec-597bbbc7b4d7 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:05:51.794745 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 588/1150] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Aug 31 01:05:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:05:51.800402 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcde2683-d1d6-450c-a229-70870b9d2e44 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:05:51.899959 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcde2683-d1d6-450c-a229-70870b9d2e44 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:05:51.933206 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd3e9b7-2665-4eba-87aa-f908abbde533 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:05:51.933828 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd3e9b7-2665-4eba-87aa-f908abbde533 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:51.934109 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd3e9b7-2665-4eba-87aa-f908abbde533 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:51.934384 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd3e9b7-2665-4eba-87aa-f908abbde533 None admin] PATH_INFO: `/v3/users/59ed194b0ff2451fb7b1f6c60e5e0801` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:51.935514 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd3e9b7-2665-4eba-87aa-f908abbde533 None admin] RBAC: Authorizing `identity:get_user(user_id=59ed194b0ff2451fb7b1f6c60e5e0801)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:05:51.936969 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd3e9b7-2665-4eba-87aa-f908abbde533 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:05:51.938537 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 563/1151] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Aug 31 01:05:51 2022] GET /identity/v3/users/59ed194b0ff2451fb7b1f6c60e5e0801 => generated 286 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:05:51.944293 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c75522f-575b-4965-8bec-597bbbc7b4d7 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:05:52.007859 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:05:52.042277 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c75522f-575b-4965-8bec-597bbbc7b4d7 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:05:52.073881 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88cb0b2-3771-459f-98b7-332c0a0f3877 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:05:52.074495 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88cb0b2-3771-459f-98b7-332c0a0f3877 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:52.074776 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88cb0b2-3771-459f-98b7-332c0a0f3877 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:52.075315 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88cb0b2-3771-459f-98b7-332c0a0f3877 None admin] PATH_INFO: `/v3/projects/d70a08d6795b4b3fa382f511754d30cb` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:52.076419 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88cb0b2-3771-459f-98b7-332c0a0f3877 None admin] RBAC: Authorizing `identity:get_project(project_id=d70a08d6795b4b3fa382f511754d30cb)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:05:52.077876 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88cb0b2-3771-459f-98b7-332c0a0f3877 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:05:52.079460 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 589/1152] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Aug 31 01:05:51 2022] GET /identity/v3/projects/d70a08d6795b4b3fa382f511754d30cb => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:05:52.085937 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd3e9b7-2665-4eba-87aa-f908abbde533 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:05:52.130917 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:05:52.182262 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd3e9b7-2665-4eba-87aa-f908abbde533 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:05:52.215810 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998381a6-3347-40ba-a306-191ed7ac80fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:05:52.216454 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998381a6-3347-40ba-a306-191ed7ac80fc None admin] REQUEST_METHOD: `PUT` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:52.216759 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998381a6-3347-40ba-a306-191ed7ac80fc None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:52.217042 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998381a6-3347-40ba-a306-191ed7ac80fc None admin] PATH_INFO: `/v3/projects/d70a08d6795b4b3fa382f511754d30cb/users/59ed194b0ff2451fb7b1f6c60e5e0801/roles/d009411496b8452b88e77ffb8db015a7` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:52.218924 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998381a6-3347-40ba-a306-191ed7ac80fc None admin] RBAC: Authorizing `identity:create_grant(project_id=d70a08d6795b4b3fa382f511754d30cb, user_id=59ed194b0ff2451fb7b1f6c60e5e0801, role_id=d009411496b8452b88e77ffb8db015a7)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:05:52.220348 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998381a6-3347-40ba-a306-191ed7ac80fc None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:05:52.306886 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 564/1153] 10.222.0.20 () {58 vars in 1439 bytes} [Wed Aug 31 01:05:52 2022] PUT /identity/v3/projects/d70a08d6795b4b3fa382f511754d30cb/users/59ed194b0ff2451fb7b1f6c60e5e0801/roles/d009411496b8452b88e77ffb8db015a7 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:05:54.162732 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd211c9-809d-4b67-b908-22217ee80024 None None] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:54.163239 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd211c9-809d-4b67-b908-22217ee80024 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:54.163239 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd211c9-809d-4b67-b908-22217ee80024 None None] PATH_INFO: `/` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:54.164531 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 590/1154] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:05:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:05:54.169866 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076cc14d-9ddf-417b-8abe-ec2759e2f53f None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:54.170264 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076cc14d-9ddf-417b-8abe-ec2759e2f53f None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:54.170423 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076cc14d-9ddf-417b-8abe-ec2759e2f53f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:54.188557 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-076cc14d-9ddf-417b-8abe-ec2759e2f53f None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:05:54.189761 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-076cc14d-9ddf-417b-8abe-ec2759e2f53f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:05:54.253867 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 565/1155] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:05:54 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 01:05:54.521238 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd211c9-809d-4b67-b908-22217ee80024 None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:05:54.619775 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd211c9-809d-4b67-b908-22217ee80024 None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:05:54.651419 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be814b6b-452c-4f11-a8a3-532073fb44a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:05:54.652036 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be814b6b-452c-4f11-a8a3-532073fb44a7 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:54.652318 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be814b6b-452c-4f11-a8a3-532073fb44a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:54.652592 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be814b6b-452c-4f11-a8a3-532073fb44a7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:54.654419 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:05:54.663655 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be814b6b-452c-4f11-a8a3-532073fb44a7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 01:05:54.668470 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application [None req-be814b6b-452c-4f11-a8a3-532073fb44a7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 01:05:54.668470 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:05:54.668470 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:05:54.668470 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:05:54.668470 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:05:54.668470 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:05:54.668470 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:05:54.668470 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:05:54.668470 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:05:54.668470 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:05:54.668470 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:05:54.668470 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:05:54.668470 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 01:05:54.668470 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 01:05:54.668470 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 01:05:54.668470 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 01:05:54.673112 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 01:05:54.673112 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 01:05:54.673112 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 01:05:54.673112 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 01:05:54.673112 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:05:54.673112 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:05:54.673112 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:05:54.673112 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:05:54.673112 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:05:54.673112 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:05:54.673112 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:05:54.673112 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application with Lock( Aug 31 01:05:54.673112 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:05:54.673112 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:05:54.673112 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:05:54.673112 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:05:54.673962 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:05:54.673962 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:05:54.673962 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:05:54.673962 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:05:54.673962 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 01:05:54.673962 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 01:05:54.673962 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 01:05:54.673962 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 01:05:54.673962 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 01:05:54.673962 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 01:05:54.673962 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 01:05:54.673962 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Aug 31 01:05:54.673962 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 591/1156] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Aug 31 01:05:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:05:54.675472 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076cc14d-9ddf-417b-8abe-ec2759e2f53f None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:05:54.709317 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:05:54.772486 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076cc14d-9ddf-417b-8abe-ec2759e2f53f None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:05:54.804759 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788d345c-bae7-4fe9-a785-9e564d5ff8c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:05:54.805350 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788d345c-bae7-4fe9-a785-9e564d5ff8c7 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:54.805628 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788d345c-bae7-4fe9-a785-9e564d5ff8c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:54.805898 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788d345c-bae7-4fe9-a785-9e564d5ff8c7 None admin] PATH_INFO: `/v3/roles` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:54.806547 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788d345c-bae7-4fe9-a785-9e564d5ff8c7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:05:54.806846 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788d345c-bae7-4fe9-a785-9e564d5ff8c7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:05:54.808199 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788d345c-bae7-4fe9-a785-9e564d5ff8c7 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:05:54.815847 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 566/1157] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Aug 31 01:05:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:05:54.821528 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be814b6b-452c-4f11-a8a3-532073fb44a7 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:05:54.921518 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be814b6b-452c-4f11-a8a3-532073fb44a7 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:05:54.953724 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657b8c5d-9df3-4ced-ba98-41e38b10c719 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:05:54.954346 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657b8c5d-9df3-4ced-ba98-41e38b10c719 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:54.954624 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657b8c5d-9df3-4ced-ba98-41e38b10c719 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:54.954896 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657b8c5d-9df3-4ced-ba98-41e38b10c719 None admin] PATH_INFO: `/v3/users/59ed194b0ff2451fb7b1f6c60e5e0801` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:54.955998 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657b8c5d-9df3-4ced-ba98-41e38b10c719 None admin] RBAC: Authorizing `identity:get_user(user_id=59ed194b0ff2451fb7b1f6c60e5e0801)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:05:54.957989 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657b8c5d-9df3-4ced-ba98-41e38b10c719 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:05:54.959551 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 592/1158] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Aug 31 01:05:54 2022] GET /identity/v3/users/59ed194b0ff2451fb7b1f6c60e5e0801 => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:05:54.965459 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788d345c-bae7-4fe9-a785-9e564d5ff8c7 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:05:55.057074 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:05:55.062555 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788d345c-bae7-4fe9-a785-9e564d5ff8c7 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:05:55.095692 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7219efb3-4576-44f8-b369-a5be665d0557 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:05:55.096290 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7219efb3-4576-44f8-b369-a5be665d0557 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:55.096576 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7219efb3-4576-44f8-b369-a5be665d0557 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:55.096886 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7219efb3-4576-44f8-b369-a5be665d0557 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:55.161824 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7219efb3-4576-44f8-b369-a5be665d0557 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:05:55.163272 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7219efb3-4576-44f8-b369-a5be665d0557 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:05:55.251727 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:05:55.258335 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 567/1159] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Aug 31 01:05:54 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 01:05:55.264420 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657b8c5d-9df3-4ced-ba98-41e38b10c719 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:05:55.323775 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:05:55.361300 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657b8c5d-9df3-4ced-ba98-41e38b10c719 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:05:55.392871 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df1b809-ce1d-40af-89d3-00362afb8491 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:05:55.393501 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df1b809-ce1d-40af-89d3-00362afb8491 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:55.393762 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df1b809-ce1d-40af-89d3-00362afb8491 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:55.394034 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df1b809-ce1d-40af-89d3-00362afb8491 None admin] PATH_INFO: `/v3/projects/d70a08d6795b4b3fa382f511754d30cb` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:55.395678 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df1b809-ce1d-40af-89d3-00362afb8491 None admin] RBAC: Authorizing `identity:get_project(project_id=d70a08d6795b4b3fa382f511754d30cb)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:05:55.397114 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df1b809-ce1d-40af-89d3-00362afb8491 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:05:55.398702 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 593/1160] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Aug 31 01:05:55 2022] GET /identity/v3/projects/d70a08d6795b4b3fa382f511754d30cb => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:05:55.404755 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7219efb3-4576-44f8-b369-a5be665d0557 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:05:55.498803 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7219efb3-4576-44f8-b369-a5be665d0557 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:05:55.531100 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a75aa7-2446-458c-988c-887ce5111e77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:05:55.531688 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a75aa7-2446-458c-988c-887ce5111e77 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:55.531967 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a75aa7-2446-458c-988c-887ce5111e77 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:55.532240 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a75aa7-2446-458c-988c-887ce5111e77 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:55.532972 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a75aa7-2446-458c-988c-887ce5111e77 None admin] RBAC: Adding query filter params (role.id=d009411496b8452b88e77ffb8db015a7, scope.project.id=d70a08d6795b4b3fa382f511754d30cb, user.id=59ed194b0ff2451fb7b1f6c60e5e0801) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:05:55.533279 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a75aa7-2446-458c-988c-887ce5111e77 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:05:55.534694 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a75aa7-2446-458c-988c-887ce5111e77 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:05:55.540207 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 568/1161] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Aug 31 01:05:55 2022] GET /identity/v3/role_assignments?user.id=59ed194b0ff2451fb7b1f6c60e5e0801&scope.project.id=d70a08d6795b4b3fa382f511754d30cb&role.id=d009411496b8452b88e77ffb8db015a7 => generated 608 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:05:57.433120 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477f2e72-626f-439b-ba57-de3ab6b52fc8 None None] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:57.433647 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477f2e72-626f-439b-ba57-de3ab6b52fc8 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:57.433647 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477f2e72-626f-439b-ba57-de3ab6b52fc8 None None] PATH_INFO: `/` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:57.434984 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 594/1162] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:05:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:05:57.439989 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cff7e7-37d6-4f01-a8a1-6e6314fb5142 None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:57.440395 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cff7e7-37d6-4f01-a8a1-6e6314fb5142 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:57.440550 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cff7e7-37d6-4f01-a8a1-6e6314fb5142 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:57.459645 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-03cff7e7-37d6-4f01-a8a1-6e6314fb5142 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:05:57.460863 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-03cff7e7-37d6-4f01-a8a1-6e6314fb5142 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:05:57.517397 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:05:57.528808 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 569/1163] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:05:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 01:05:57.797693 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477f2e72-626f-439b-ba57-de3ab6b52fc8 None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:05:57.896480 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477f2e72-626f-439b-ba57-de3ab6b52fc8 None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:05:57.923042 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:05:57.929104 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab12199-eae9-4266-9bf0-d9809fc51d2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:05:57.929712 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab12199-eae9-4266-9bf0-d9809fc51d2e None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:57.929995 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab12199-eae9-4266-9bf0-d9809fc51d2e None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:57.936516 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab12199-eae9-4266-9bf0-d9809fc51d2e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:57.994886 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab12199-eae9-4266-9bf0-d9809fc51d2e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:05:57.996322 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab12199-eae9-4266-9bf0-d9809fc51d2e None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:05:58.089953 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 595/1164] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Aug 31 01:05:57 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 01:05:58.095783 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cff7e7-37d6-4f01-a8a1-6e6314fb5142 None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:05:58.191140 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cff7e7-37d6-4f01-a8a1-6e6314fb5142 None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:05:58.224095 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5471b4cc-e3d6-4942-8474-8dc6abcf1aaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:05:58.224710 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5471b4cc-e3d6-4942-8474-8dc6abcf1aaf None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:58.225020 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5471b4cc-e3d6-4942-8474-8dc6abcf1aaf None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:58.225295 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5471b4cc-e3d6-4942-8474-8dc6abcf1aaf None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:58.231663 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5471b4cc-e3d6-4942-8474-8dc6abcf1aaf None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:05:58.233138 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5471b4cc-e3d6-4942-8474-8dc6abcf1aaf None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:05:58.244851 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application [None req-5471b4cc-e3d6-4942-8474-8dc6abcf1aaf None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 31 01:05:58.244851 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:05:58.244851 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 31 01:05:58.244851 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 31 01:05:58.244851 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 01:05:58.244851 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 01:05:58.244851 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 01:05:58.244851 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 01:05:58.244851 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Aug 31 01:05:58.244851 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Aug 31 01:05:58.244851 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 01:05:58.244851 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Aug 31 01:05:58.244851 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:05:58.244851 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:05:58.244851 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:05:58.244851 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:05:58.244851 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:05:58.246052 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:05:58.246052 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:05:58.246052 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:05:58.246052 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:05:58.246052 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:05:58.246052 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:05:58.246052 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 01:05:58.246052 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 01:05:58.246052 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 01:05:58.246052 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 01:05:58.246052 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:05:58.246052 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:05:58.246052 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:05:58.246052 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:05:58.246052 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:05:58.246052 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:05:58.247112 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:05:58.247112 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application with Lock( Aug 31 01:05:58.247112 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:05:58.247112 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:05:58.247112 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:05:58.247112 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:05:58.247112 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:05:58.247112 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:05:58.247112 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:05:58.247112 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:05:58.247112 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 31 01:05:58.247112 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 01:05:58.247112 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 31 01:05:58.247112 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Aug 31 01:05:58.247112 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 570/1165] 10.222.0.20 () {56 vars in 1117 bytes} [Wed Aug 31 01:05:58 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:05:58.251673 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab12199-eae9-4266-9bf0-d9809fc51d2e None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:05:58.279793 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:05:58.345876 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab12199-eae9-4266-9bf0-d9809fc51d2e None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:05:58.378427 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5029159-e77a-4e38-ae0c-a42a67d45551 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:05:58.379040 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5029159-e77a-4e38-ae0c-a42a67d45551 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:58.379318 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5029159-e77a-4e38-ae0c-a42a67d45551 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:58.379588 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5029159-e77a-4e38-ae0c-a42a67d45551 None admin] PATH_INFO: `/v3/domains` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:58.380212 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5029159-e77a-4e38-ae0c-a42a67d45551 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:05:58.380510 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5029159-e77a-4e38-ae0c-a42a67d45551 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:05:58.381877 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5029159-e77a-4e38-ae0c-a42a67d45551 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:05:58.388613 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 596/1166] 10.222.0.20 () {56 vars in 1115 bytes} [Wed Aug 31 01:05:58 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:06:00.245674 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21cec3f-7de1-4f96-b333-fa28a65cc3fa None None] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:06:00.246218 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21cec3f-7de1-4f96-b333-fa28a65cc3fa None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:06:00.246218 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21cec3f-7de1-4f96-b333-fa28a65cc3fa None None] PATH_INFO: `/` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:06:00.247556 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 571/1167] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:06:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:06:00.252786 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97772d48-c440-474e-b9a6-9f3add088a9e None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:06:00.253209 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97772d48-c440-474e-b9a6-9f3add088a9e None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:06:00.253340 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97772d48-c440-474e-b9a6-9f3add088a9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:06:00.271465 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-97772d48-c440-474e-b9a6-9f3add088a9e None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:06:00.272636 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-97772d48-c440-474e-b9a6-9f3add088a9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:06:00.336119 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 597/1168] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:06:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 01:06:00.605128 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21cec3f-7de1-4f96-b333-fa28a65cc3fa None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:06:00.655014 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:06:00.703427 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21cec3f-7de1-4f96-b333-fa28a65cc3fa None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:06:00.736260 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034ddb5c-e631-4372-880e-6e2cf8847960 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:06:00.736882 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034ddb5c-e631-4372-880e-6e2cf8847960 None admin] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:06:00.737162 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034ddb5c-e631-4372-880e-6e2cf8847960 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:06:00.737435 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034ddb5c-e631-4372-880e-6e2cf8847960 None admin] PATH_INFO: `/v3/domains` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:06:00.738723 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034ddb5c-e631-4372-880e-6e2cf8847960 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:06:00.740585 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034ddb5c-e631-4372-880e-6e2cf8847960 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:06:00.817855 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 572/1169] 10.222.0.20 () {60 vars in 1138 bytes} [Wed Aug 31 01:06:00 2022] POST /identity/v3/domains => generated 267 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:06:02.738454 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79db10f3-32a3-46cf-8144-78bfb0e6d8a2 None None] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:06:02.739015 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79db10f3-32a3-46cf-8144-78bfb0e6d8a2 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:06:02.739015 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79db10f3-32a3-46cf-8144-78bfb0e6d8a2 None None] PATH_INFO: `/` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:06:02.740354 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 598/1170] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:06:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:06:02.745525 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15455c09-deff-4099-947a-efc5e4eee212 None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:06:02.745943 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15455c09-deff-4099-947a-efc5e4eee212 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:06:02.746084 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15455c09-deff-4099-947a-efc5e4eee212 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:06:02.764508 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-15455c09-deff-4099-947a-efc5e4eee212 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:06:02.765735 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-15455c09-deff-4099-947a-efc5e4eee212 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:06:02.830442 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 573/1171] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:06:02 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 01:06:03.107803 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79db10f3-32a3-46cf-8144-78bfb0e6d8a2 None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:06:03.133372 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:06:03.206960 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79db10f3-32a3-46cf-8144-78bfb0e6d8a2 None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:06:03.238746 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7af7c3-3941-4dce-a58e-ca117dc078b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:06:03.239360 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7af7c3-3941-4dce-a58e-ca117dc078b3 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:06:03.239634 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7af7c3-3941-4dce-a58e-ca117dc078b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:06:03.239901 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7af7c3-3941-4dce-a58e-ca117dc078b3 None admin] PATH_INFO: `/v3/domains/ad56f5750c234a35a47aab34045bb3e9` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:06:03.249819 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7af7c3-3941-4dce-a58e-ca117dc078b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=ad56f5750c234a35a47aab34045bb3e9)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:06:03.251249 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7af7c3-3941-4dce-a58e-ca117dc078b3 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:06:03.252810 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 599/1172] 10.222.0.20 () {56 vars in 1183 bytes} [Wed Aug 31 01:06:03 2022] GET /identity/v3/domains/ad56f5750c234a35a47aab34045bb3e9 => generated 267 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:06:03.259122 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15455c09-deff-4099-947a-efc5e4eee212 None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:06:03.299660 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:06:03.355372 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15455c09-deff-4099-947a-efc5e4eee212 None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:06:03.389435 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0004cfd-1a85-442d-b5b7-f73b41824ad0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:06:03.389977 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0004cfd-1a85-442d-b5b7-f73b41824ad0 None admin] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:06:03.390256 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0004cfd-1a85-442d-b5b7-f73b41824ad0 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:06:03.390530 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0004cfd-1a85-442d-b5b7-f73b41824ad0 None admin] PATH_INFO: `/v3/projects` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:06:03.391180 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0004cfd-1a85-442d-b5b7-f73b41824ad0 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:06:03.392612 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0004cfd-1a85-442d-b5b7-f73b41824ad0 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:06:03.479073 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 574/1173] 10.222.0.20 () {60 vars in 1141 bytes} [Wed Aug 31 01:06:03 2022] POST /identity/v3/projects => generated 361 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:06:05.385497 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0038b70d-8cef-44a3-a278-e5e58f3227be None None] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:06:05.386046 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0038b70d-8cef-44a3-a278-e5e58f3227be None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:06:05.386046 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0038b70d-8cef-44a3-a278-e5e58f3227be None None] PATH_INFO: `/` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:06:05.387396 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 600/1174] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:06:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:06:05.392607 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596ebbb6-3b1a-4978-bcf9-8ae3db08ea64 None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:06:05.393055 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596ebbb6-3b1a-4978-bcf9-8ae3db08ea64 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:06:05.393199 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596ebbb6-3b1a-4978-bcf9-8ae3db08ea64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:06:05.410834 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-596ebbb6-3b1a-4978-bcf9-8ae3db08ea64 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:06:05.412003 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-596ebbb6-3b1a-4978-bcf9-8ae3db08ea64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:06:05.475133 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 575/1175] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:06:05 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 01:06:05.739617 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0038b70d-8cef-44a3-a278-e5e58f3227be None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:06:05.823332 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:06:05.839508 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0038b70d-8cef-44a3-a278-e5e58f3227be None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:06:05.872211 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd024cf1-fec0-4194-a02f-2f99842b1180 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:06:05.872856 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd024cf1-fec0-4194-a02f-2f99842b1180 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:06:05.873133 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd024cf1-fec0-4194-a02f-2f99842b1180 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:06:05.873401 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd024cf1-fec0-4194-a02f-2f99842b1180 None admin] PATH_INFO: `/v3/domains/ad56f5750c234a35a47aab34045bb3e9` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:06:05.874493 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd024cf1-fec0-4194-a02f-2f99842b1180 None admin] RBAC: Authorizing `identity:get_domain(domain_id=ad56f5750c234a35a47aab34045bb3e9)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:06:05.875902 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd024cf1-fec0-4194-a02f-2f99842b1180 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:06:05.877508 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 601/1176] 10.222.0.20 () {56 vars in 1183 bytes} [Wed Aug 31 01:06:05 2022] GET /identity/v3/domains/ad56f5750c234a35a47aab34045bb3e9 => generated 267 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:06:05.883231 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596ebbb6-3b1a-4978-bcf9-8ae3db08ea64 None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:06:05.906758 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:06:05.981371 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596ebbb6-3b1a-4978-bcf9-8ae3db08ea64 None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:06:06.015273 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fd7f07-2981-471e-ad10-b631567cfa05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:06:06.015822 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fd7f07-2981-471e-ad10-b631567cfa05 None admin] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:06:06.016113 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fd7f07-2981-471e-ad10-b631567cfa05 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:06:06.016391 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fd7f07-2981-471e-ad10-b631567cfa05 None admin] PATH_INFO: `/v3/users` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:06:06.017070 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fd7f07-2981-471e-ad10-b631567cfa05 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:06:06.018505 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fd7f07-2981-471e-ad10-b631567cfa05 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:06:06.108786 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 576/1177] 10.222.0.20 () {60 vars in 1132 bytes} [Wed Aug 31 01:06:05 2022] POST /identity/v3/users => generated 311 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:06:08.021197 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f4e681-6d9d-4c16-91a2-e609363418f7 None None] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:06:08.021724 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f4e681-6d9d-4c16-91a2-e609363418f7 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:06:08.021724 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f4e681-6d9d-4c16-91a2-e609363418f7 None None] PATH_INFO: `/` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:06:08.023070 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 602/1178] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:06:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:06:08.028133 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c254d3-07ff-42cd-afdc-9e15089073d5 None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:06:08.028519 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c254d3-07ff-42cd-afdc-9e15089073d5 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:06:08.028689 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c254d3-07ff-42cd-afdc-9e15089073d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:06:08.046466 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-19c254d3-07ff-42cd-afdc-9e15089073d5 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:06:08.047645 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-19c254d3-07ff-42cd-afdc-9e15089073d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:06:08.112078 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 577/1179] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:06:08 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 01:06:08.378065 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f4e681-6d9d-4c16-91a2-e609363418f7 None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:06:08.475282 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f4e681-6d9d-4c16-91a2-e609363418f7 None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:06:08.508075 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8b8b13-d932-4e94-92a9-fbf2e8296d2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:06:08.508684 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8b8b13-d932-4e94-92a9-fbf2e8296d2d None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:06:08.508996 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8b8b13-d932-4e94-92a9-fbf2e8296d2d None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:06:08.509269 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8b8b13-d932-4e94-92a9-fbf2e8296d2d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:06:08.519091 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8b8b13-d932-4e94-92a9-fbf2e8296d2d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 01:06:08.523908 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application [None req-ae8b8b13-d932-4e94-92a9-fbf2e8296d2d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 01:06:08.523908 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:06:08.523908 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:06:08.523908 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:06:08.523908 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:06:08.523908 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:06:08.523908 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:06:08.523908 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:06:08.523908 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:06:08.523908 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:06:08.523908 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:06:08.523908 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:06:08.523908 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 01:06:08.523908 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 01:06:08.523908 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 01:06:08.523908 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 01:06:08.525142 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 01:06:08.525142 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 01:06:08.525142 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 01:06:08.525142 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 01:06:08.525142 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:06:08.525142 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:06:08.525142 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:06:08.525142 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:06:08.525142 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:06:08.525142 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:06:08.525142 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:06:08.525142 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application with Lock( Aug 31 01:06:08.525142 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:06:08.525142 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:06:08.525142 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:06:08.525142 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:06:08.526102 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:06:08.526102 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:06:08.526102 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:06:08.526102 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:06:08.526102 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 01:06:08.526102 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 01:06:08.526102 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 01:06:08.526102 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 01:06:08.526102 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 01:06:08.526102 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 01:06:08.526102 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 01:06:08.526102 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Aug 31 01:06:08.526102 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 603/1180] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Aug 31 01:06:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:06:08.530881 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c254d3-07ff-42cd-afdc-9e15089073d5 None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:06:08.540947 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:06:08.626509 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c254d3-07ff-42cd-afdc-9e15089073d5 None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:06:08.658150 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddd1267-54a1-4003-828c-97eb89d42f0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:06:08.658686 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddd1267-54a1-4003-828c-97eb89d42f0b None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:06:08.658963 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddd1267-54a1-4003-828c-97eb89d42f0b None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:06:08.659235 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddd1267-54a1-4003-828c-97eb89d42f0b None admin] PATH_INFO: `/v3/roles` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:06:08.660400 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddd1267-54a1-4003-828c-97eb89d42f0b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:06:08.660701 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddd1267-54a1-4003-828c-97eb89d42f0b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:06:08.662097 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddd1267-54a1-4003-828c-97eb89d42f0b None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:06:08.669806 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 578/1181] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Aug 31 01:06:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:06:08.676121 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8b8b13-d932-4e94-92a9-fbf2e8296d2d None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:06:08.690964 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:06:08.772115 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8b8b13-d932-4e94-92a9-fbf2e8296d2d None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:06:08.804873 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deae817a-ffed-4e39-a852-a8d6719fd365 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:06:08.805484 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deae817a-ffed-4e39-a852-a8d6719fd365 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:06:08.805765 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deae817a-ffed-4e39-a852-a8d6719fd365 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:06:08.806040 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deae817a-ffed-4e39-a852-a8d6719fd365 None admin] PATH_INFO: `/v3/users/9dcf1d9cf495434b88e03e9cdbac299d` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:06:08.818529 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deae817a-ffed-4e39-a852-a8d6719fd365 None admin] RBAC: Authorizing `identity:get_user(user_id=9dcf1d9cf495434b88e03e9cdbac299d)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:06:08.819998 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deae817a-ffed-4e39-a852-a8d6719fd365 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:06:08.821613 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 604/1182] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Aug 31 01:06:08 2022] GET /identity/v3/users/9dcf1d9cf495434b88e03e9cdbac299d => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:06:08.827629 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddd1267-54a1-4003-828c-97eb89d42f0b None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:06:08.896415 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:06:08.929223 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddd1267-54a1-4003-828c-97eb89d42f0b None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:06:08.962201 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b868525-ca7b-45fc-ae18-e33c4bf26a09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:06:08.962839 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b868525-ca7b-45fc-ae18-e33c4bf26a09 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:06:08.963116 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b868525-ca7b-45fc-ae18-e33c4bf26a09 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:06:08.963387 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b868525-ca7b-45fc-ae18-e33c4bf26a09 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:06:09.028491 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b868525-ca7b-45fc-ae18-e33c4bf26a09 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:06:09.029961 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b868525-ca7b-45fc-ae18-e33c4bf26a09 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:06:09.093604 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:06:09.123939 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 579/1183] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Aug 31 01:06:08 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 01:06:09.129907 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deae817a-ffed-4e39-a852-a8d6719fd365 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:06:09.196872 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:06:09.227306 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deae817a-ffed-4e39-a852-a8d6719fd365 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:06:09.259390 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d40426-4dcc-46ac-b069-601f428cccd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:06:09.260005 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d40426-4dcc-46ac-b069-601f428cccd7 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:06:09.260287 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d40426-4dcc-46ac-b069-601f428cccd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:06:09.260559 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d40426-4dcc-46ac-b069-601f428cccd7 None admin] PATH_INFO: `/v3/projects/eac60302cfd64aa2a9cb9c872ebae349` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:06:09.261685 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d40426-4dcc-46ac-b069-601f428cccd7 None admin] RBAC: Authorizing `identity:get_project(project_id=eac60302cfd64aa2a9cb9c872ebae349)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:06:09.263105 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d40426-4dcc-46ac-b069-601f428cccd7 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:06:09.264696 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 605/1184] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Aug 31 01:06:09 2022] GET /identity/v3/projects/eac60302cfd64aa2a9cb9c872ebae349 => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:06:09.271228 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b868525-ca7b-45fc-ae18-e33c4bf26a09 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:06:09.366882 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b868525-ca7b-45fc-ae18-e33c4bf26a09 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:06:09.400400 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6ab118-7416-4d1b-a257-efeb8a672307 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:06:09.401017 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6ab118-7416-4d1b-a257-efeb8a672307 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:06:09.401301 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6ab118-7416-4d1b-a257-efeb8a672307 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:06:09.401581 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6ab118-7416-4d1b-a257-efeb8a672307 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:06:09.402296 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6ab118-7416-4d1b-a257-efeb8a672307 None admin] RBAC: Adding query filter params (role.id=d009411496b8452b88e77ffb8db015a7, scope.project.id=eac60302cfd64aa2a9cb9c872ebae349, user.id=9dcf1d9cf495434b88e03e9cdbac299d) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:06:09.402609 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6ab118-7416-4d1b-a257-efeb8a672307 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:06:09.404026 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6ab118-7416-4d1b-a257-efeb8a672307 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:06:09.409219 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 580/1185] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Aug 31 01:06:09 2022] GET /identity/v3/role_assignments?user.id=9dcf1d9cf495434b88e03e9cdbac299d&scope.project.id=eac60302cfd64aa2a9cb9c872ebae349&role.id=d009411496b8452b88e77ffb8db015a7 => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:06:11.277964 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b2a0ed-c138-464f-af13-4b249331c49f None None] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:06:11.278507 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b2a0ed-c138-464f-af13-4b249331c49f None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:06:11.278507 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b2a0ed-c138-464f-af13-4b249331c49f None None] PATH_INFO: `/` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:06:11.279844 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 606/1186] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:06:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:06:11.285679 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb72e8c9-3869-4e65-b969-f45fd0e7f7f7 None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:06:11.286107 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb72e8c9-3869-4e65-b969-f45fd0e7f7f7 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:06:11.286219 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb72e8c9-3869-4e65-b969-f45fd0e7f7f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:06:11.303950 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-fb72e8c9-3869-4e65-b969-f45fd0e7f7f7 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:06:11.305179 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-fb72e8c9-3869-4e65-b969-f45fd0e7f7f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:06:11.330937 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:06:11.371112 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 581/1187] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:06:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 01:06:11.635078 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b2a0ed-c138-464f-af13-4b249331c49f None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:06:11.732164 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b2a0ed-c138-464f-af13-4b249331c49f None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:06:11.764017 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b557b10d-357d-4f3b-a9cb-d98a44090114 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:06:11.764621 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b557b10d-357d-4f3b-a9cb-d98a44090114 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:06:11.764926 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b557b10d-357d-4f3b-a9cb-d98a44090114 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:06:11.765197 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b557b10d-357d-4f3b-a9cb-d98a44090114 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:06:11.772144 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:06:11.775733 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b557b10d-357d-4f3b-a9cb-d98a44090114 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 01:06:11.780486 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application [None req-b557b10d-357d-4f3b-a9cb-d98a44090114 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 01:06:11.780486 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:06:11.780486 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:06:11.780486 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:06:11.780486 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:06:11.780486 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:06:11.780486 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:06:11.780486 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:06:11.780486 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:06:11.780486 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:06:11.780486 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:06:11.780486 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:06:11.780486 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 01:06:11.780486 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 01:06:11.780486 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 01:06:11.780486 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 01:06:11.788466 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 01:06:11.788466 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 01:06:11.788466 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 01:06:11.788466 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 01:06:11.788466 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:06:11.788466 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:06:11.788466 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:06:11.788466 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:06:11.788466 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:06:11.788466 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:06:11.788466 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:06:11.788466 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application with Lock( Aug 31 01:06:11.788466 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:06:11.788466 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:06:11.788466 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:06:11.788466 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:06:11.789517 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:06:11.789517 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:06:11.789517 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:06:11.789517 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:06:11.789517 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 01:06:11.789517 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 01:06:11.789517 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 01:06:11.789517 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 01:06:11.789517 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 01:06:11.789517 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 01:06:11.789517 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 01:06:11.789517 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Aug 31 01:06:11.789517 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 607/1188] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Aug 31 01:06:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:06:11.790340 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb72e8c9-3869-4e65-b969-f45fd0e7f7f7 None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:06:11.884739 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb72e8c9-3869-4e65-b969-f45fd0e7f7f7 None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:06:11.919707 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de309b3b-d17c-4a6e-a21c-fbd5e3adb2c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:06:11.920312 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de309b3b-d17c-4a6e-a21c-fbd5e3adb2c4 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:06:11.920592 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de309b3b-d17c-4a6e-a21c-fbd5e3adb2c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:06:11.920893 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de309b3b-d17c-4a6e-a21c-fbd5e3adb2c4 None admin] PATH_INFO: `/v3/roles` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:06:11.921546 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de309b3b-d17c-4a6e-a21c-fbd5e3adb2c4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:06:11.921847 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de309b3b-d17c-4a6e-a21c-fbd5e3adb2c4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:06:11.923219 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de309b3b-d17c-4a6e-a21c-fbd5e3adb2c4 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:06:11.930941 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 582/1189] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Aug 31 01:06:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:06:11.936733 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b557b10d-357d-4f3b-a9cb-d98a44090114 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:06:12.033649 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b557b10d-357d-4f3b-a9cb-d98a44090114 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:06:12.066789 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15933be2-9978-46cd-b0f5-606f691882e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:06:12.067396 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15933be2-9978-46cd-b0f5-606f691882e0 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:06:12.067676 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15933be2-9978-46cd-b0f5-606f691882e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:06:12.067947 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15933be2-9978-46cd-b0f5-606f691882e0 None admin] PATH_INFO: `/v3/users/9dcf1d9cf495434b88e03e9cdbac299d` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:06:12.069086 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15933be2-9978-46cd-b0f5-606f691882e0 None admin] RBAC: Authorizing `identity:get_user(user_id=9dcf1d9cf495434b88e03e9cdbac299d)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:06:12.070508 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15933be2-9978-46cd-b0f5-606f691882e0 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:06:12.072337 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 608/1190] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Aug 31 01:06:11 2022] GET /identity/v3/users/9dcf1d9cf495434b88e03e9cdbac299d => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:06:12.078174 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de309b3b-d17c-4a6e-a21c-fbd5e3adb2c4 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:06:12.105224 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:06:12.174820 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de309b3b-d17c-4a6e-a21c-fbd5e3adb2c4 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:06:12.207975 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54acaad8-d705-44fa-8290-971592ec382a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:06:12.208593 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54acaad8-d705-44fa-8290-971592ec382a None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:06:12.208917 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54acaad8-d705-44fa-8290-971592ec382a None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:06:12.209196 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54acaad8-d705-44fa-8290-971592ec382a None admin] PATH_INFO: `/v3/projects/eac60302cfd64aa2a9cb9c872ebae349` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:06:12.210299 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54acaad8-d705-44fa-8290-971592ec382a None admin] RBAC: Authorizing `identity:get_project(project_id=eac60302cfd64aa2a9cb9c872ebae349)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:06:12.211731 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54acaad8-d705-44fa-8290-971592ec382a None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:06:12.213388 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 583/1191] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Aug 31 01:06:12 2022] GET /identity/v3/projects/eac60302cfd64aa2a9cb9c872ebae349 => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:06:12.219189 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15933be2-9978-46cd-b0f5-606f691882e0 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:06:12.282834 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:06:12.316522 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15933be2-9978-46cd-b0f5-606f691882e0 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:06:12.349282 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4828c79-a91f-4cdb-86e9-9c6ce915bfc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:06:12.349928 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4828c79-a91f-4cdb-86e9-9c6ce915bfc2 None admin] REQUEST_METHOD: `PUT` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:06:12.350207 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4828c79-a91f-4cdb-86e9-9c6ce915bfc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:06:12.350484 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4828c79-a91f-4cdb-86e9-9c6ce915bfc2 None admin] PATH_INFO: `/v3/projects/eac60302cfd64aa2a9cb9c872ebae349/users/9dcf1d9cf495434b88e03e9cdbac299d/roles/d009411496b8452b88e77ffb8db015a7` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:06:12.352380 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4828c79-a91f-4cdb-86e9-9c6ce915bfc2 None admin] RBAC: Authorizing `identity:create_grant(project_id=eac60302cfd64aa2a9cb9c872ebae349, user_id=9dcf1d9cf495434b88e03e9cdbac299d, role_id=d009411496b8452b88e77ffb8db015a7)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:06:12.354063 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4828c79-a91f-4cdb-86e9-9c6ce915bfc2 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:06:12.415848 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 609/1192] 10.222.0.20 () {58 vars in 1439 bytes} [Wed Aug 31 01:06:12 2022] PUT /identity/v3/projects/eac60302cfd64aa2a9cb9c872ebae349/users/9dcf1d9cf495434b88e03e9cdbac299d/roles/d009411496b8452b88e77ffb8db015a7 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:06:14.290976 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67374038-5d0c-4b4f-9bc9-c7820f72a25e None None] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:06:14.291492 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67374038-5d0c-4b4f-9bc9-c7820f72a25e None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:06:14.291583 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67374038-5d0c-4b4f-9bc9-c7820f72a25e None None] PATH_INFO: `/` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:06:14.292880 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 584/1193] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:06:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:06:14.298035 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91358cca-0c76-4087-884a-17504f9a003b None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:06:14.298452 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91358cca-0c76-4087-884a-17504f9a003b None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:06:14.298592 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91358cca-0c76-4087-884a-17504f9a003b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:06:14.317408 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-91358cca-0c76-4087-884a-17504f9a003b None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:06:14.318587 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-91358cca-0c76-4087-884a-17504f9a003b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:06:14.382548 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 610/1194] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:06:14 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 01:06:14.652418 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67374038-5d0c-4b4f-9bc9-c7820f72a25e None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:06:14.743827 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:06:14.750737 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67374038-5d0c-4b4f-9bc9-c7820f72a25e None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:06:14.782765 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302e7759-c64d-422f-88a2-5d8ecb628e9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:06:14.783393 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302e7759-c64d-422f-88a2-5d8ecb628e9d None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:06:14.783675 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302e7759-c64d-422f-88a2-5d8ecb628e9d None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:06:14.783954 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302e7759-c64d-422f-88a2-5d8ecb628e9d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:06:14.794736 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302e7759-c64d-422f-88a2-5d8ecb628e9d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 01:06:14.799617 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application [None req-302e7759-c64d-422f-88a2-5d8ecb628e9d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 01:06:14.799617 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:06:14.799617 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:06:14.799617 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:06:14.799617 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:06:14.799617 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:06:14.799617 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:06:14.799617 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:06:14.799617 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:06:14.799617 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:06:14.799617 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:06:14.799617 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:06:14.799617 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 01:06:14.799617 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 01:06:14.799617 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 01:06:14.799617 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 01:06:14.801288 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 01:06:14.801288 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 01:06:14.801288 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 01:06:14.801288 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 01:06:14.801288 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:06:14.801288 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:06:14.801288 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:06:14.801288 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:06:14.801288 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:06:14.801288 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:06:14.801288 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:06:14.801288 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application with Lock( Aug 31 01:06:14.801288 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:06:14.801288 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:06:14.801288 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:06:14.801288 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:06:14.802336 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:06:14.802336 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:06:14.802336 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:06:14.802336 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:06:14.802336 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 01:06:14.802336 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 01:06:14.802336 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 01:06:14.802336 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 01:06:14.802336 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 01:06:14.802336 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 01:06:14.802336 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 01:06:14.802336 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Aug 31 01:06:14.805101 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 585/1195] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Aug 31 01:06:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:06:14.807294 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91358cca-0c76-4087-884a-17504f9a003b None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:06:14.862874 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:06:14.904747 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91358cca-0c76-4087-884a-17504f9a003b None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:06:14.939493 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78091328-6e2b-4861-8978-53b99332a475 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:06:14.940093 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78091328-6e2b-4861-8978-53b99332a475 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:06:14.940376 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78091328-6e2b-4861-8978-53b99332a475 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:06:14.940651 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78091328-6e2b-4861-8978-53b99332a475 None admin] PATH_INFO: `/v3/roles` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:06:14.941425 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78091328-6e2b-4861-8978-53b99332a475 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:06:14.941729 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78091328-6e2b-4861-8978-53b99332a475 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:06:14.943093 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78091328-6e2b-4861-8978-53b99332a475 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:06:14.951346 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 611/1196] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Aug 31 01:06:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:06:14.957838 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302e7759-c64d-422f-88a2-5d8ecb628e9d None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:06:15.054710 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302e7759-c64d-422f-88a2-5d8ecb628e9d None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:06:15.088602 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2107f11e-d1a4-47ef-8b6b-8791d17ae8a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:06:15.089254 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2107f11e-d1a4-47ef-8b6b-8791d17ae8a3 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:06:15.089549 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2107f11e-d1a4-47ef-8b6b-8791d17ae8a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:06:15.089823 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2107f11e-d1a4-47ef-8b6b-8791d17ae8a3 None admin] PATH_INFO: `/v3/users/9dcf1d9cf495434b88e03e9cdbac299d` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:06:15.090953 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2107f11e-d1a4-47ef-8b6b-8791d17ae8a3 None admin] RBAC: Authorizing `identity:get_user(user_id=9dcf1d9cf495434b88e03e9cdbac299d)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:06:15.092392 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2107f11e-d1a4-47ef-8b6b-8791d17ae8a3 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:06:15.093991 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 586/1197] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Aug 31 01:06:14 2022] GET /identity/v3/users/9dcf1d9cf495434b88e03e9cdbac299d => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:06:15.100252 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78091328-6e2b-4861-8978-53b99332a475 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:06:15.195818 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78091328-6e2b-4861-8978-53b99332a475 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:06:15.212662 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:06:15.229539 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91300e1c-25b2-42a1-9fc3-ec57b7649a5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:06:15.230135 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91300e1c-25b2-42a1-9fc3-ec57b7649a5b None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:06:15.230416 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91300e1c-25b2-42a1-9fc3-ec57b7649a5b None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:06:15.230708 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91300e1c-25b2-42a1-9fc3-ec57b7649a5b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:06:15.295597 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91300e1c-25b2-42a1-9fc3-ec57b7649a5b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:06:15.297059 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91300e1c-25b2-42a1-9fc3-ec57b7649a5b None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:06:15.391140 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 612/1198] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Aug 31 01:06:15 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 01:06:15.397745 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2107f11e-d1a4-47ef-8b6b-8791d17ae8a3 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:06:15.421806 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:06:15.495022 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2107f11e-d1a4-47ef-8b6b-8791d17ae8a3 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:06:15.527956 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c90a0e-57b9-48a9-8704-0cc5b3c4c281 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:06:15.528574 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c90a0e-57b9-48a9-8704-0cc5b3c4c281 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:06:15.528887 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c90a0e-57b9-48a9-8704-0cc5b3c4c281 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:06:15.529167 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c90a0e-57b9-48a9-8704-0cc5b3c4c281 None admin] PATH_INFO: `/v3/projects/eac60302cfd64aa2a9cb9c872ebae349` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:06:15.530276 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c90a0e-57b9-48a9-8704-0cc5b3c4c281 None admin] RBAC: Authorizing `identity:get_project(project_id=eac60302cfd64aa2a9cb9c872ebae349)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:06:15.531699 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c90a0e-57b9-48a9-8704-0cc5b3c4c281 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:06:15.533333 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 587/1199] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Aug 31 01:06:15 2022] GET /identity/v3/projects/eac60302cfd64aa2a9cb9c872ebae349 => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:06:15.539223 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91300e1c-25b2-42a1-9fc3-ec57b7649a5b None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:06:15.556606 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:06:15.634872 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91300e1c-25b2-42a1-9fc3-ec57b7649a5b None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:06:15.667476 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbee681e-b3d2-433a-905b-66cf2ec09cc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:06:15.668067 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbee681e-b3d2-433a-905b-66cf2ec09cc6 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:06:15.668346 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbee681e-b3d2-433a-905b-66cf2ec09cc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:06:15.668618 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbee681e-b3d2-433a-905b-66cf2ec09cc6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:06:15.669360 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbee681e-b3d2-433a-905b-66cf2ec09cc6 None admin] RBAC: Adding query filter params (role.id=d009411496b8452b88e77ffb8db015a7, scope.project.id=eac60302cfd64aa2a9cb9c872ebae349, user.id=9dcf1d9cf495434b88e03e9cdbac299d) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:06:15.669669 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbee681e-b3d2-433a-905b-66cf2ec09cc6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:06:15.671077 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbee681e-b3d2-433a-905b-66cf2ec09cc6 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:06:15.677160 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 613/1200] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Aug 31 01:06:15 2022] GET /identity/v3/role_assignments?user.id=9dcf1d9cf495434b88e03e9cdbac299d&scope.project.id=eac60302cfd64aa2a9cb9c872ebae349&role.id=d009411496b8452b88e77ffb8db015a7 => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:06:25.523929 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=55 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:06:25.674099 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=59 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:07:29.151709 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0347b3-ff9b-4cf9-aca0-78d56dc63949 None None] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:07:29.152282 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0347b3-ff9b-4cf9-aca0-78d56dc63949 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:07:29.152282 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0347b3-ff9b-4cf9-aca0-78d56dc63949 None None] PATH_INFO: `/` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:07:29.153582 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 588/1201] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:07:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:07:29.159249 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c857be-f234-4083-a656-55233de706d1 None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:07:29.159810 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c857be-f234-4083-a656-55233de706d1 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:07:29.159810 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c857be-f234-4083-a656-55233de706d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:07:29.177569 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-25c857be-f234-4083-a656-55233de706d1 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:07:29.178787 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-25c857be-f234-4083-a656-55233de706d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:07:29.242855 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 614/1202] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:07:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 01:07:29.517843 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0347b3-ff9b-4cf9-aca0-78d56dc63949 None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:07:29.617747 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0347b3-ff9b-4cf9-aca0-78d56dc63949 None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:07:29.650742 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43066940-9cc8-4644-a28a-5fa8449740f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:07:29.651361 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43066940-9cc8-4644-a28a-5fa8449740f7 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:07:29.651643 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43066940-9cc8-4644-a28a-5fa8449740f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:07:29.651919 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43066940-9cc8-4644-a28a-5fa8449740f7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:07:29.662660 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43066940-9cc8-4644-a28a-5fa8449740f7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:07:29.664304 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43066940-9cc8-4644-a28a-5fa8449740f7 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:07:29.680277 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application [None req-43066940-9cc8-4644-a28a-5fa8449740f7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 01:07:29.680277 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:07:29.680277 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:07:29.680277 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:07:29.680277 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:07:29.680277 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:07:29.680277 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:07:29.680277 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:07:29.680277 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:07:29.680277 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:07:29.680277 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:07:29.680277 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:07:29.680277 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 01:07:29.680277 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 01:07:29.680277 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 01:07:29.680277 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 01:07:29.682772 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:07:29.682772 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:07:29.682772 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:07:29.682772 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:07:29.682772 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:07:29.682772 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:07:29.682772 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:07:29.682772 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application with Lock( Aug 31 01:07:29.682772 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:07:29.682772 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:07:29.682772 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:07:29.682772 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:07:29.682772 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:07:29.682772 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:07:29.682772 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:07:29.682772 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:07:29.682772 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 01:07:29.684971 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 01:07:29.684971 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 01:07:29.684971 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Aug 31 01:07:29.684971 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 589/1203] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 01:07:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:07:29.689437 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c857be-f234-4083-a656-55233de706d1 None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:07:29.785549 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c857be-f234-4083-a656-55233de706d1 None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:07:29.813436 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:07:29.819092 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1350e98-3e83-49cd-813d-9cde760aa204 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:07:29.819695 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1350e98-3e83-49cd-813d-9cde760aa204 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:07:29.819975 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1350e98-3e83-49cd-813d-9cde760aa204 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:07:29.820246 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1350e98-3e83-49cd-813d-9cde760aa204 None admin] PATH_INFO: `/v3/domains` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:07:29.820907 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1350e98-3e83-49cd-813d-9cde760aa204 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:07:29.821212 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1350e98-3e83-49cd-813d-9cde760aa204 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:07:29.822570 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1350e98-3e83-49cd-813d-9cde760aa204 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:07:29.833648 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 615/1204] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 31 01:07:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:07:29.840036 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43066940-9cc8-4644-a28a-5fa8449740f7 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:07:29.894087 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:07:29.944199 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43066940-9cc8-4644-a28a-5fa8449740f7 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:07:29.977691 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8124e8f-126b-4c9c-a800-00faad7c4831 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:07:29.978292 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8124e8f-126b-4c9c-a800-00faad7c4831 None admin] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:07:29.978571 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8124e8f-126b-4c9c-a800-00faad7c4831 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:07:29.978845 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8124e8f-126b-4c9c-a800-00faad7c4831 None admin] PATH_INFO: `/v3/users` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:07:29.979493 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8124e8f-126b-4c9c-a800-00faad7c4831 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:07:29.980948 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8124e8f-126b-4c9c-a800-00faad7c4831 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:07:30.047478 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 590/1205] 10.222.0.20 () {60 vars in 1132 bytes} [Wed Aug 31 01:07:29 2022] POST /identity/v3/users => generated 251 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:07:32.004643 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a22b07-c685-49ab-b311-0adf3fd96393 None None] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:07:32.005240 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a22b07-c685-49ab-b311-0adf3fd96393 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:07:32.005240 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a22b07-c685-49ab-b311-0adf3fd96393 None None] PATH_INFO: `/` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:07:32.006557 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 616/1206] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:07:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:07:32.012449 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d33bf1-f164-466e-b04f-eb3bb13c6a99 None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:07:32.012888 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d33bf1-f164-466e-b04f-eb3bb13c6a99 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:07:32.013043 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d33bf1-f164-466e-b04f-eb3bb13c6a99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:07:32.030827 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-05d33bf1-f164-466e-b04f-eb3bb13c6a99 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:07:32.034016 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-05d33bf1-f164-466e-b04f-eb3bb13c6a99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:07:32.098084 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 591/1207] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:07:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 01:07:32.372505 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a22b07-c685-49ab-b311-0adf3fd96393 None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:07:32.472629 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a22b07-c685-49ab-b311-0adf3fd96393 None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:07:32.505376 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4915bb8-f7a9-4390-bbac-eaeac7268f92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:07:32.505996 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4915bb8-f7a9-4390-bbac-eaeac7268f92 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:07:32.506276 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4915bb8-f7a9-4390-bbac-eaeac7268f92 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:07:32.506549 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4915bb8-f7a9-4390-bbac-eaeac7268f92 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:07:32.516694 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4915bb8-f7a9-4390-bbac-eaeac7268f92 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 01:07:32.521538 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application [None req-c4915bb8-f7a9-4390-bbac-eaeac7268f92 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 01:07:32.521538 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:07:32.521538 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:07:32.521538 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:07:32.521538 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:07:32.521538 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:07:32.521538 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:07:32.521538 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:07:32.521538 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:07:32.521538 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:07:32.521538 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:07:32.521538 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:07:32.521538 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 01:07:32.521538 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 01:07:32.521538 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 01:07:32.521538 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 01:07:32.527666 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 01:07:32.527666 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 01:07:32.527666 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 01:07:32.527666 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 01:07:32.527666 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:07:32.527666 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:07:32.527666 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:07:32.527666 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:07:32.527666 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:07:32.527666 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:07:32.527666 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:07:32.527666 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application with Lock( Aug 31 01:07:32.527666 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:07:32.527666 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:07:32.527666 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:07:32.527666 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:07:32.528783 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:07:32.528783 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:07:32.528783 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:07:32.528783 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:07:32.528783 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 01:07:32.528783 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 01:07:32.528783 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 01:07:32.528783 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 01:07:32.528783 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 01:07:32.528783 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 01:07:32.528783 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 01:07:32.528783 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Aug 31 01:07:32.528783 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 617/1208] 10.222.0.20 () {56 vars in 1102 bytes} [Wed Aug 31 01:07:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:07:32.530747 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d33bf1-f164-466e-b04f-eb3bb13c6a99 None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:07:32.569435 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:07:32.626309 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d33bf1-f164-466e-b04f-eb3bb13c6a99 None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:07:32.659495 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a62986-0128-4cc5-a899-61404ca2d3e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:07:32.670519 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a62986-0128-4cc5-a899-61404ca2d3e8 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:07:32.670519 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a62986-0128-4cc5-a899-61404ca2d3e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:07:32.670519 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a62986-0128-4cc5-a899-61404ca2d3e8 None admin] PATH_INFO: `/v3/roles` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:07:32.670519 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a62986-0128-4cc5-a899-61404ca2d3e8 None admin] RBAC: Adding query filter params (name=service) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:07:32.670519 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a62986-0128-4cc5-a899-61404ca2d3e8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:07:32.670519 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a62986-0128-4cc5-a899-61404ca2d3e8 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:07:32.671221 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 592/1209] 10.222.0.20 () {56 vars in 1103 bytes} [Wed Aug 31 01:07:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:07:32.677013 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4915bb8-f7a9-4390-bbac-eaeac7268f92 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:07:32.726060 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:07:32.776170 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4915bb8-f7a9-4390-bbac-eaeac7268f92 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:07:32.809659 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf895f0-d933-4c87-83dc-fd77262a5470 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:07:32.810284 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf895f0-d933-4c87-83dc-fd77262a5470 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:07:32.810568 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf895f0-d933-4c87-83dc-fd77262a5470 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:07:32.810846 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf895f0-d933-4c87-83dc-fd77262a5470 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:07:32.821046 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf895f0-d933-4c87-83dc-fd77262a5470 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:07:32.822588 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf895f0-d933-4c87-83dc-fd77262a5470 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:07:32.836994 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application [None req-dbf895f0-d933-4c87-83dc-fd77262a5470 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 01:07:32.836994 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:07:32.836994 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:07:32.836994 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:07:32.836994 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:07:32.836994 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:07:32.836994 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:07:32.836994 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:07:32.836994 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:07:32.836994 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:07:32.836994 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:07:32.836994 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:07:32.836994 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 01:07:32.836994 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 01:07:32.836994 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 01:07:32.836994 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 01:07:32.838598 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:07:32.838598 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:07:32.838598 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:07:32.838598 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:07:32.838598 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:07:32.838598 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:07:32.838598 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:07:32.838598 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application with Lock( Aug 31 01:07:32.838598 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:07:32.838598 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:07:32.838598 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:07:32.838598 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:07:32.838598 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:07:32.838598 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:07:32.838598 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:07:32.838598 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:07:32.838598 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 01:07:32.839806 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 01:07:32.839806 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 01:07:32.839806 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Aug 31 01:07:32.839806 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 618/1210] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 01:07:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:07:32.844549 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a62986-0128-4cc5-a899-61404ca2d3e8 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:07:32.942220 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:07:32.945087 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a62986-0128-4cc5-a899-61404ca2d3e8 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:07:32.977738 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43acfe52-3d0e-43a0-96c9-280b5a5b8e77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:07:32.978866 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43acfe52-3d0e-43a0-96c9-280b5a5b8e77 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:07:32.979140 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43acfe52-3d0e-43a0-96c9-280b5a5b8e77 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:07:32.979411 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43acfe52-3d0e-43a0-96c9-280b5a5b8e77 None admin] PATH_INFO: `/v3/domains` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:07:32.980044 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43acfe52-3d0e-43a0-96c9-280b5a5b8e77 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:07:32.980345 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43acfe52-3d0e-43a0-96c9-280b5a5b8e77 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:07:32.981738 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43acfe52-3d0e-43a0-96c9-280b5a5b8e77 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:07:32.993414 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 593/1211] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 31 01:07:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:07:32.999318 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf895f0-d933-4c87-83dc-fd77262a5470 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:07:33.098526 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf895f0-d933-4c87-83dc-fd77262a5470 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:07:33.100073 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:07:33.132023 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f6d9f7-237e-4e39-aec7-33c85fa7b510 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:07:33.132647 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f6d9f7-237e-4e39-aec7-33c85fa7b510 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:07:33.132959 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f6d9f7-237e-4e39-aec7-33c85fa7b510 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:07:33.133232 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f6d9f7-237e-4e39-aec7-33c85fa7b510 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:07:33.141146 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f6d9f7-237e-4e39-aec7-33c85fa7b510 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:07:33.145277 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f6d9f7-237e-4e39-aec7-33c85fa7b510 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:07:33.154232 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application [None req-d7f6d9f7-237e-4e39-aec7-33c85fa7b510 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 31 01:07:33.154232 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:07:33.154232 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:07:33.154232 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:07:33.154232 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:07:33.154232 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:07:33.154232 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:07:33.154232 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:07:33.154232 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:07:33.154232 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:07:33.154232 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:07:33.154232 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:07:33.154232 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 01:07:33.154232 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 01:07:33.154232 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 01:07:33.154232 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 01:07:33.155363 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:07:33.155363 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:07:33.155363 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 01:07:33.155363 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 01:07:33.155363 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 01:07:33.155363 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 01:07:33.155363 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:07:33.155363 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:07:33.155363 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:07:33.155363 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:07:33.155363 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:07:33.155363 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application with Lock( Aug 31 01:07:33.155363 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:07:33.155363 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:07:33.155363 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:07:33.155363 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:07:33.155363 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:07:33.155363 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:07:33.156350 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:07:33.156350 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:07:33.156350 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 01:07:33.156350 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 01:07:33.156350 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 01:07:33.156350 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 01:07:33.156350 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 01:07:33.156350 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 01:07:33.156350 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 31 01:07:33.156350 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Aug 31 01:07:33.156350 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 619/1212] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 01:07:32 2022] GET /identity/v3/users/placement => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:07:33.161488 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43acfe52-3d0e-43a0-96c9-280b5a5b8e77 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:07:33.261433 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43acfe52-3d0e-43a0-96c9-280b5a5b8e77 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:07:33.294546 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d50f66-01d1-41ed-bcde-27d5b90502dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:07:33.295145 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d50f66-01d1-41ed-bcde-27d5b90502dc None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:07:33.295424 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d50f66-01d1-41ed-bcde-27d5b90502dc None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:07:33.295697 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d50f66-01d1-41ed-bcde-27d5b90502dc None admin] PATH_INFO: `/v3/users` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:07:33.296412 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d50f66-01d1-41ed-bcde-27d5b90502dc None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:07:33.296761 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d50f66-01d1-41ed-bcde-27d5b90502dc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:07:33.298338 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d50f66-01d1-41ed-bcde-27d5b90502dc None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:07:33.314942 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 594/1213] 10.222.0.20 () {56 vars in 1143 bytes} [Wed Aug 31 01:07:33 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:07:33.320956 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f6d9f7-237e-4e39-aec7-33c85fa7b510 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:07:33.420316 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f6d9f7-237e-4e39-aec7-33c85fa7b510 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:07:33.453223 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8256c7c0-1164-477c-a738-3d2d60aad4b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:07:33.453826 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8256c7c0-1164-477c-a738-3d2d60aad4b3 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:07:33.454112 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8256c7c0-1164-477c-a738-3d2d60aad4b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:07:33.454390 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8256c7c0-1164-477c-a738-3d2d60aad4b3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:07:33.484168 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:07:33.521327 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8256c7c0-1164-477c-a738-3d2d60aad4b3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:07:33.522803 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8256c7c0-1164-477c-a738-3d2d60aad4b3 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:07:33.620679 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 620/1214] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Aug 31 01:07:33 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 01:07:33.626559 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d50f66-01d1-41ed-bcde-27d5b90502dc None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:07:33.646595 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:07:33.725146 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d50f66-01d1-41ed-bcde-27d5b90502dc None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:07:33.757115 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb70cc4-e250-40dc-b799-667403704292 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:07:33.757734 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb70cc4-e250-40dc-b799-667403704292 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:07:33.758039 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb70cc4-e250-40dc-b799-667403704292 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:07:33.758311 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb70cc4-e250-40dc-b799-667403704292 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:07:33.769396 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb70cc4-e250-40dc-b799-667403704292 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:07:33.770849 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb70cc4-e250-40dc-b799-667403704292 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:07:33.785676 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application [None req-0cb70cc4-e250-40dc-b799-667403704292 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 01:07:33.785676 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:07:33.785676 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:07:33.785676 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:07:33.785676 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:07:33.785676 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:07:33.785676 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:07:33.785676 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:07:33.785676 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:07:33.785676 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:07:33.785676 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:07:33.785676 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:07:33.785676 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 01:07:33.785676 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 01:07:33.785676 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 01:07:33.785676 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 01:07:33.786827 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:07:33.786827 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:07:33.786827 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:07:33.786827 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:07:33.786827 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:07:33.786827 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:07:33.786827 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:07:33.786827 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application with Lock( Aug 31 01:07:33.786827 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:07:33.786827 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:07:33.786827 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:07:33.786827 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:07:33.786827 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:07:33.786827 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:07:33.786827 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:07:33.786827 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:07:33.786827 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 01:07:33.787924 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 01:07:33.787924 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 01:07:33.787924 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Aug 31 01:07:33.787924 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 595/1215] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 01:07:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 01:07:33.794114 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8256c7c0-1164-477c-a738-3d2d60aad4b3 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:07:33.852375 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:07:33.889519 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8256c7c0-1164-477c-a738-3d2d60aad4b3 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:07:33.923171 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa42935-5c1a-4c88-8c23-cdc39f806879 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:07:33.923781 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa42935-5c1a-4c88-8c23-cdc39f806879 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:07:33.924063 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa42935-5c1a-4c88-8c23-cdc39f806879 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:07:33.924336 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa42935-5c1a-4c88-8c23-cdc39f806879 None admin] PATH_INFO: `/v3/domains` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:07:33.925000 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa42935-5c1a-4c88-8c23-cdc39f806879 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:07:33.925323 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa42935-5c1a-4c88-8c23-cdc39f806879 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:07:33.926683 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa42935-5c1a-4c88-8c23-cdc39f806879 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:07:33.938116 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 621/1216] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 31 01:07:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:07:33.943833 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb70cc4-e250-40dc-b799-667403704292 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:07:34.013176 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:07:34.042936 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb70cc4-e250-40dc-b799-667403704292 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:07:34.074987 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54a0d15-86d4-4853-938e-8c02c59d9434 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:07:34.075633 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54a0d15-86d4-4853-938e-8c02c59d9434 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:07:34.075940 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54a0d15-86d4-4853-938e-8c02c59d9434 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:07:34.076211 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54a0d15-86d4-4853-938e-8c02c59d9434 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:07:34.083164 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54a0d15-86d4-4853-938e-8c02c59d9434 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:07:34.084596 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54a0d15-86d4-4853-938e-8c02c59d9434 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:07:34.095602 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application [None req-d54a0d15-86d4-4853-938e-8c02c59d9434 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 01:07:34.095602 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:07:34.095602 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:07:34.095602 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:07:34.095602 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:07:34.095602 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:07:34.095602 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:07:34.095602 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:07:34.095602 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:07:34.095602 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:07:34.095602 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:07:34.095602 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:07:34.095602 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 01:07:34.095602 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 01:07:34.095602 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 01:07:34.096702 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 01:07:34.096702 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:07:34.096702 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:07:34.096702 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:07:34.096702 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:07:34.096702 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:07:34.096702 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:07:34.096702 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:07:34.096702 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application with Lock( Aug 31 01:07:34.096702 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:07:34.096702 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:07:34.096702 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:07:34.096702 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:07:34.096702 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:07:34.096702 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:07:34.096702 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:07:34.096702 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:07:34.097818 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 01:07:34.097818 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 01:07:34.097818 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 01:07:34.097818 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 01:07:34.097818 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 01:07:34.097818 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 01:07:34.097818 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 01:07:34.097818 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Aug 31 01:07:34.097818 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 596/1217] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Aug 31 01:07:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:07:34.102602 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa42935-5c1a-4c88-8c23-cdc39f806879 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:07:34.201201 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa42935-5c1a-4c88-8c23-cdc39f806879 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:07:34.223157 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:07:34.234894 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90839ae-9b78-48bc-8d25-e30a8028fe7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:07:34.235492 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90839ae-9b78-48bc-8d25-e30a8028fe7f None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:07:34.235776 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90839ae-9b78-48bc-8d25-e30a8028fe7f None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:07:34.236047 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90839ae-9b78-48bc-8d25-e30a8028fe7f None admin] PATH_INFO: `/v3/projects` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:07:34.236738 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90839ae-9b78-48bc-8d25-e30a8028fe7f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:07:34.237060 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90839ae-9b78-48bc-8d25-e30a8028fe7f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:07:34.238468 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90839ae-9b78-48bc-8d25-e30a8028fe7f None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:07:34.249942 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 622/1218] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Aug 31 01:07:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:07:34.256116 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54a0d15-86d4-4853-938e-8c02c59d9434 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:07:34.358547 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54a0d15-86d4-4853-938e-8c02c59d9434 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:07:34.385778 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:07:34.391402 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75697fc-c1b2-495d-bae7-41b29f0cd71a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:07:34.391998 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75697fc-c1b2-495d-bae7-41b29f0cd71a None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:07:34.392280 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75697fc-c1b2-495d-bae7-41b29f0cd71a None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:07:34.392552 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75697fc-c1b2-495d-bae7-41b29f0cd71a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:07:34.393305 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75697fc-c1b2-495d-bae7-41b29f0cd71a None admin] RBAC: Adding query filter params (role.id=7d00a4cfa13744a69604ac2cf1a5816e, scope.project.id=60b2abc15c2849ba8a1ed92117cc981b, user.id=dea0fe5b1d6b405cb47b95192580303b) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:07:34.393615 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75697fc-c1b2-495d-bae7-41b29f0cd71a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:07:34.395070 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75697fc-c1b2-495d-bae7-41b29f0cd71a None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:07:34.400828 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 597/1219] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Aug 31 01:07:34 2022] GET /identity/v3/role_assignments?user.id=dea0fe5b1d6b405cb47b95192580303b&scope.project.id=60b2abc15c2849ba8a1ed92117cc981b&role.id=7d00a4cfa13744a69604ac2cf1a5816e => generated 259 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:07:36.279390 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bd0f8d-56c8-4fe1-bfd9-e769951ad880 None None] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:07:36.280000 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bd0f8d-56c8-4fe1-bfd9-e769951ad880 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:07:36.280000 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bd0f8d-56c8-4fe1-bfd9-e769951ad880 None None] PATH_INFO: `/` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:07:36.281293 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 623/1220] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:07:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:07:36.286237 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33a9b6b-0957-496e-a427-eb3d8fa297e4 None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:07:36.286630 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33a9b6b-0957-496e-a427-eb3d8fa297e4 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:07:36.286798 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33a9b6b-0957-496e-a427-eb3d8fa297e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:07:36.304680 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-b33a9b6b-0957-496e-a427-eb3d8fa297e4 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:07:36.305892 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-b33a9b6b-0957-496e-a427-eb3d8fa297e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:07:36.375470 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 598/1221] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:07:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 01:07:36.641907 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bd0f8d-56c8-4fe1-bfd9-e769951ad880 None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:07:36.740390 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bd0f8d-56c8-4fe1-bfd9-e769951ad880 None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:07:36.774213 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cae8071-e5ba-4d91-a692-78f24ddd0f7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:07:36.774831 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cae8071-e5ba-4d91-a692-78f24ddd0f7a None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:07:36.775108 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cae8071-e5ba-4d91-a692-78f24ddd0f7a None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:07:36.775377 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cae8071-e5ba-4d91-a692-78f24ddd0f7a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:07:36.785168 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cae8071-e5ba-4d91-a692-78f24ddd0f7a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 01:07:36.789991 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application [None req-8cae8071-e5ba-4d91-a692-78f24ddd0f7a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 01:07:36.789991 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:07:36.789991 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:07:36.789991 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:07:36.789991 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:07:36.789991 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:07:36.789991 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:07:36.789991 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:07:36.789991 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:07:36.789991 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:07:36.789991 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:07:36.789991 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:07:36.789991 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 01:07:36.789991 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 01:07:36.789991 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 01:07:36.789991 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 01:07:36.791192 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 01:07:36.791192 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 01:07:36.791192 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 01:07:36.791192 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 01:07:36.791192 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:07:36.791192 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:07:36.791192 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:07:36.791192 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:07:36.791192 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:07:36.791192 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:07:36.791192 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:07:36.791192 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application with Lock( Aug 31 01:07:36.791192 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:07:36.791192 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:07:36.791192 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:07:36.791192 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:07:36.792323 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:07:36.792323 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:07:36.792323 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:07:36.792323 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:07:36.792323 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 01:07:36.792323 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 01:07:36.792323 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 01:07:36.792323 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 01:07:36.792323 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 01:07:36.792323 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 01:07:36.792323 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 01:07:36.792323 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Aug 31 01:07:36.792323 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 624/1222] 10.222.0.20 () {56 vars in 1102 bytes} [Wed Aug 31 01:07:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:07:36.797059 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33a9b6b-0957-496e-a427-eb3d8fa297e4 None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:07:36.896108 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33a9b6b-0957-496e-a427-eb3d8fa297e4 None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:07:36.919750 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:07:36.929987 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcb9339-fb5c-4f37-9aa4-ff409d1d2f45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:07:36.930591 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcb9339-fb5c-4f37-9aa4-ff409d1d2f45 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:07:36.930871 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcb9339-fb5c-4f37-9aa4-ff409d1d2f45 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:07:36.931771 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcb9339-fb5c-4f37-9aa4-ff409d1d2f45 None admin] PATH_INFO: `/v3/roles` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:07:36.932434 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcb9339-fb5c-4f37-9aa4-ff409d1d2f45 None admin] RBAC: Adding query filter params (name=service) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:07:36.932759 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcb9339-fb5c-4f37-9aa4-ff409d1d2f45 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:07:36.934132 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcb9339-fb5c-4f37-9aa4-ff409d1d2f45 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:07:36.941824 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 599/1223] 10.222.0.20 () {56 vars in 1103 bytes} [Wed Aug 31 01:07:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:07:36.947600 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cae8071-e5ba-4d91-a692-78f24ddd0f7a None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:07:36.990684 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:07:37.046088 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cae8071-e5ba-4d91-a692-78f24ddd0f7a None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:07:37.078994 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85ad7fa-8198-4e65-9e0c-d10abd25989d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:07:37.079616 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85ad7fa-8198-4e65-9e0c-d10abd25989d None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:07:37.079940 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85ad7fa-8198-4e65-9e0c-d10abd25989d None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:07:37.080220 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85ad7fa-8198-4e65-9e0c-d10abd25989d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:07:37.090028 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85ad7fa-8198-4e65-9e0c-d10abd25989d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:07:37.091470 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85ad7fa-8198-4e65-9e0c-d10abd25989d None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:07:37.106963 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application [None req-c85ad7fa-8198-4e65-9e0c-d10abd25989d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 01:07:37.106963 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:07:37.106963 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:07:37.106963 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:07:37.106963 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:07:37.106963 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:07:37.106963 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:07:37.106963 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:07:37.106963 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:07:37.106963 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:07:37.106963 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:07:37.106963 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:07:37.106963 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 01:07:37.106963 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 01:07:37.106963 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 01:07:37.106963 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 01:07:37.107968 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:07:37.107968 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:07:37.107968 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:07:37.107968 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:07:37.107968 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:07:37.107968 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:07:37.107968 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:07:37.107968 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application with Lock( Aug 31 01:07:37.107968 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:07:37.107968 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:07:37.107968 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:07:37.107968 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:07:37.107968 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:07:37.107968 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:07:37.107968 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:07:37.107968 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:07:37.107968 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 01:07:37.109015 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 01:07:37.109015 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 01:07:37.109015 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Aug 31 01:07:37.109015 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 625/1224] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 01:07:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:07:37.114117 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcb9339-fb5c-4f37-9aa4-ff409d1d2f45 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:07:37.213162 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcb9339-fb5c-4f37-9aa4-ff409d1d2f45 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:07:37.247349 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0de3be-7d81-4918-beca-2db296e03c23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:07:37.247958 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0de3be-7d81-4918-beca-2db296e03c23 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:07:37.248242 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0de3be-7d81-4918-beca-2db296e03c23 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:07:37.248521 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0de3be-7d81-4918-beca-2db296e03c23 None admin] PATH_INFO: `/v3/domains` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:07:37.249193 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0de3be-7d81-4918-beca-2db296e03c23 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:07:37.249500 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0de3be-7d81-4918-beca-2db296e03c23 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:07:37.250869 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0de3be-7d81-4918-beca-2db296e03c23 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:07:37.262852 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 600/1225] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 31 01:07:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:07:37.268504 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85ad7fa-8198-4e65-9e0c-d10abd25989d None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:07:37.360857 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:07:37.367361 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85ad7fa-8198-4e65-9e0c-d10abd25989d None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:07:37.400141 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4d3c21-1037-4b6d-a2ba-ba7cd8bc37f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:07:37.400819 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4d3c21-1037-4b6d-a2ba-ba7cd8bc37f2 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:07:37.401100 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4d3c21-1037-4b6d-a2ba-ba7cd8bc37f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:07:37.401373 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4d3c21-1037-4b6d-a2ba-ba7cd8bc37f2 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:07:37.408562 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4d3c21-1037-4b6d-a2ba-ba7cd8bc37f2 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:07:37.410020 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4d3c21-1037-4b6d-a2ba-ba7cd8bc37f2 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:07:37.421879 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application [None req-1f4d3c21-1037-4b6d-a2ba-ba7cd8bc37f2 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 31 01:07:37.421879 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:07:37.421879 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:07:37.421879 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:07:37.421879 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:07:37.421879 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:07:37.421879 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:07:37.421879 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:07:37.421879 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:07:37.421879 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:07:37.421879 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:07:37.421879 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:07:37.421879 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 01:07:37.421879 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 01:07:37.421879 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 01:07:37.421879 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 01:07:37.426840 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:07:37.426840 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:07:37.426840 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 01:07:37.426840 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 01:07:37.426840 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 01:07:37.426840 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 01:07:37.426840 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:07:37.426840 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:07:37.426840 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:07:37.426840 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:07:37.426840 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:07:37.426840 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application with Lock( Aug 31 01:07:37.426840 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:07:37.426840 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:07:37.426840 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:07:37.426840 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:07:37.426840 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:07:37.426840 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:07:37.427963 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:07:37.427963 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:07:37.427963 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 01:07:37.427963 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 01:07:37.427963 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 01:07:37.427963 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 01:07:37.427963 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 01:07:37.427963 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 01:07:37.427963 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 31 01:07:37.427963 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Aug 31 01:07:37.427963 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 626/1226] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 01:07:37 2022] GET /identity/v3/users/placement => generated 87 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:07:37.428857 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0de3be-7d81-4918-beca-2db296e03c23 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:07:37.475530 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:07:37.526454 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0de3be-7d81-4918-beca-2db296e03c23 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:07:37.558887 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acee860-1914-49e5-bf58-994a32b6d993 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:07:37.559478 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acee860-1914-49e5-bf58-994a32b6d993 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:07:37.559761 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acee860-1914-49e5-bf58-994a32b6d993 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:07:37.560037 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acee860-1914-49e5-bf58-994a32b6d993 None admin] PATH_INFO: `/v3/users` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:07:37.560778 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acee860-1914-49e5-bf58-994a32b6d993 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:07:37.561086 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acee860-1914-49e5-bf58-994a32b6d993 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:07:37.562513 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acee860-1914-49e5-bf58-994a32b6d993 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:07:37.580255 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 601/1227] 10.222.0.20 () {56 vars in 1143 bytes} [Wed Aug 31 01:07:37 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:07:37.586039 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4d3c21-1037-4b6d-a2ba-ba7cd8bc37f2 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:07:37.684168 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4d3c21-1037-4b6d-a2ba-ba7cd8bc37f2 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:07:37.717899 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f091e1d2-4dae-49d8-b06f-a1deef561908 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:07:37.718524 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f091e1d2-4dae-49d8-b06f-a1deef561908 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:07:37.718807 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f091e1d2-4dae-49d8-b06f-a1deef561908 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:07:37.719086 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f091e1d2-4dae-49d8-b06f-a1deef561908 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:07:37.729394 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f091e1d2-4dae-49d8-b06f-a1deef561908 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:07:37.730776 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f091e1d2-4dae-49d8-b06f-a1deef561908 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:07:37.744870 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application [None req-f091e1d2-4dae-49d8-b06f-a1deef561908 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 01:07:37.744870 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:07:37.744870 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:07:37.744870 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:07:37.744870 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:07:37.744870 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:07:37.744870 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:07:37.744870 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:07:37.744870 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:07:37.744870 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:07:37.744870 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:07:37.744870 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:07:37.744870 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 01:07:37.744870 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 01:07:37.744870 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 01:07:37.744870 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 01:07:37.746035 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:07:37.746035 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:07:37.746035 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:07:37.746035 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:07:37.746035 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:07:37.746035 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:07:37.746035 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:07:37.746035 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application with Lock( Aug 31 01:07:37.746035 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:07:37.746035 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:07:37.746035 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:07:37.746035 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:07:37.746035 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:07:37.746035 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:07:37.746035 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:07:37.746035 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:07:37.746035 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 01:07:37.747131 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 01:07:37.747131 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 01:07:37.747131 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Aug 31 01:07:37.747131 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 627/1228] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 01:07:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:07:37.751812 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acee860-1914-49e5-bf58-994a32b6d993 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:07:37.853528 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acee860-1914-49e5-bf58-994a32b6d993 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:07:37.855824 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:07:37.886546 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc074d19-2b2d-4afc-b28d-438676e90c87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:07:37.887819 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc074d19-2b2d-4afc-b28d-438676e90c87 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:07:37.887819 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc074d19-2b2d-4afc-b28d-438676e90c87 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:07:37.887819 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc074d19-2b2d-4afc-b28d-438676e90c87 None admin] PATH_INFO: `/v3/domains` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:07:37.888388 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc074d19-2b2d-4afc-b28d-438676e90c87 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:07:37.888690 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc074d19-2b2d-4afc-b28d-438676e90c87 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:07:37.890096 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc074d19-2b2d-4afc-b28d-438676e90c87 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:07:37.902222 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 602/1229] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 31 01:07:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:07:37.907887 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f091e1d2-4dae-49d8-b06f-a1deef561908 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:07:37.917026 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:07:38.004225 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f091e1d2-4dae-49d8-b06f-a1deef561908 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:07:38.037733 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a458715-0cf5-4ae5-a5c9-cd595a671f62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:07:38.038382 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a458715-0cf5-4ae5-a5c9-cd595a671f62 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:07:38.038629 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a458715-0cf5-4ae5-a5c9-cd595a671f62 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:07:38.038900 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a458715-0cf5-4ae5-a5c9-cd595a671f62 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:07:38.045322 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a458715-0cf5-4ae5-a5c9-cd595a671f62 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:07:38.046742 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a458715-0cf5-4ae5-a5c9-cd595a671f62 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:07:38.058091 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application [None req-0a458715-0cf5-4ae5-a5c9-cd595a671f62 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 01:07:38.058091 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:07:38.058091 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:07:38.058091 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:07:38.058091 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:07:38.058091 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:07:38.058091 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:07:38.058091 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:07:38.058091 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:07:38.058091 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:07:38.058091 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:07:38.058091 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:07:38.058091 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 01:07:38.058091 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 01:07:38.058091 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 01:07:38.059103 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 01:07:38.059103 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:07:38.059103 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:07:38.059103 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:07:38.059103 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:07:38.059103 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:07:38.059103 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:07:38.059103 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:07:38.059103 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application with Lock( Aug 31 01:07:38.059103 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:07:38.059103 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:07:38.059103 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:07:38.059103 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:07:38.059103 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:07:38.059103 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:07:38.059103 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:07:38.059103 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:07:38.060343 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 01:07:38.060343 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 01:07:38.060343 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 01:07:38.060343 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 01:07:38.060343 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 01:07:38.060343 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 01:07:38.060343 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 01:07:38.060343 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Aug 31 01:07:38.060343 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 628/1230] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Aug 31 01:07:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:07:38.064974 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc074d19-2b2d-4afc-b28d-438676e90c87 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:07:38.163373 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc074d19-2b2d-4afc-b28d-438676e90c87 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:07:38.198255 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46676571-d488-42c2-b68d-85489a06b3b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:07:38.198910 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46676571-d488-42c2-b68d-85489a06b3b5 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:07:38.199195 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46676571-d488-42c2-b68d-85489a06b3b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:07:38.199472 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46676571-d488-42c2-b68d-85489a06b3b5 None admin] PATH_INFO: `/v3/projects` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:07:38.200439 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46676571-d488-42c2-b68d-85489a06b3b5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:07:38.200796 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46676571-d488-42c2-b68d-85489a06b3b5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:07:38.202243 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46676571-d488-42c2-b68d-85489a06b3b5 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:07:38.214415 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 603/1231] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Aug 31 01:07:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:07:38.220387 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a458715-0cf5-4ae5-a5c9-cd595a671f62 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:07:38.284041 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:07:38.317880 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a458715-0cf5-4ae5-a5c9-cd595a671f62 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:07:38.350684 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03317c4d-ede7-45fb-984d-3eb7917e7c59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:07:38.351340 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03317c4d-ede7-45fb-984d-3eb7917e7c59 None admin] REQUEST_METHOD: `PUT` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:07:38.351612 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03317c4d-ede7-45fb-984d-3eb7917e7c59 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:07:38.351883 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03317c4d-ede7-45fb-984d-3eb7917e7c59 None admin] PATH_INFO: `/v3/projects/60b2abc15c2849ba8a1ed92117cc981b/users/dea0fe5b1d6b405cb47b95192580303b/roles/7d00a4cfa13744a69604ac2cf1a5816e` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:07:38.366661 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03317c4d-ede7-45fb-984d-3eb7917e7c59 None admin] RBAC: Authorizing `identity:create_grant(project_id=60b2abc15c2849ba8a1ed92117cc981b, user_id=dea0fe5b1d6b405cb47b95192580303b, role_id=7d00a4cfa13744a69604ac2cf1a5816e)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:07:38.369061 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03317c4d-ede7-45fb-984d-3eb7917e7c59 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:07:38.499484 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 629/1232] 10.222.0.20 () {58 vars in 1439 bytes} [Wed Aug 31 01:07:38 2022] PUT /identity/v3/projects/60b2abc15c2849ba8a1ed92117cc981b/users/dea0fe5b1d6b405cb47b95192580303b/roles/7d00a4cfa13744a69604ac2cf1a5816e => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:07:40.380676 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b39d98-2ae4-4fc4-81a4-dba3abc85de2 None None] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:07:40.381253 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b39d98-2ae4-4fc4-81a4-dba3abc85de2 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:07:40.381253 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b39d98-2ae4-4fc4-81a4-dba3abc85de2 None None] PATH_INFO: `/` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:07:40.382633 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 604/1233] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:07:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:07:40.388064 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612bfb75-36c5-4c6e-8673-a779ede3afc8 None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:07:40.388485 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612bfb75-36c5-4c6e-8673-a779ede3afc8 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:07:40.388643 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612bfb75-36c5-4c6e-8673-a779ede3afc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:07:40.406848 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-612bfb75-36c5-4c6e-8673-a779ede3afc8 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:07:40.408047 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-612bfb75-36c5-4c6e-8673-a779ede3afc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:07:40.472623 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 630/1234] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:07:40 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 01:07:40.743910 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b39d98-2ae4-4fc4-81a4-dba3abc85de2 None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:07:40.768977 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:07:40.842143 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b39d98-2ae4-4fc4-81a4-dba3abc85de2 None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:07:40.874158 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed43641-4d72-4335-a895-fa0a82796026 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:07:40.874779 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed43641-4d72-4335-a895-fa0a82796026 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:07:40.875065 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed43641-4d72-4335-a895-fa0a82796026 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:07:40.875341 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed43641-4d72-4335-a895-fa0a82796026 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:07:40.885519 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed43641-4d72-4335-a895-fa0a82796026 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 01:07:40.890275 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application [None req-1ed43641-4d72-4335-a895-fa0a82796026 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 01:07:40.890275 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:07:40.890275 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:07:40.890275 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:07:40.890275 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:07:40.890275 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:07:40.890275 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:07:40.890275 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:07:40.890275 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:07:40.890275 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:07:40.890275 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:07:40.890275 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:07:40.890275 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 01:07:40.890275 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 01:07:40.890275 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 01:07:40.890275 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 01:07:40.891311 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 01:07:40.891311 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 01:07:40.891311 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 01:07:40.891311 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 01:07:40.891311 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:07:40.891311 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:07:40.891311 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:07:40.891311 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:07:40.891311 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:07:40.891311 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:07:40.891311 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:07:40.891311 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application with Lock( Aug 31 01:07:40.891311 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:07:40.891311 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:07:40.891311 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:07:40.891311 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:07:40.892531 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:07:40.892531 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:07:40.892531 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:07:40.892531 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:07:40.892531 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 01:07:40.892531 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 01:07:40.892531 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 01:07:40.892531 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 01:07:40.892531 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 01:07:40.892531 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 01:07:40.892531 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 01:07:40.892531 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Aug 31 01:07:40.892531 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 605/1235] 10.222.0.20 () {56 vars in 1102 bytes} [Wed Aug 31 01:07:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:07:40.897509 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612bfb75-36c5-4c6e-8673-a779ede3afc8 None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:07:40.943369 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:07:40.995235 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612bfb75-36c5-4c6e-8673-a779ede3afc8 None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:07:41.028319 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bdac13-58be-4e5d-8dff-34f3ee16c733 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:07:41.029160 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bdac13-58be-4e5d-8dff-34f3ee16c733 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:07:41.029442 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bdac13-58be-4e5d-8dff-34f3ee16c733 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:07:41.029714 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bdac13-58be-4e5d-8dff-34f3ee16c733 None admin] PATH_INFO: `/v3/roles` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:07:41.030366 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bdac13-58be-4e5d-8dff-34f3ee16c733 None admin] RBAC: Adding query filter params (name=service) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:07:41.030666 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bdac13-58be-4e5d-8dff-34f3ee16c733 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:07:41.032015 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bdac13-58be-4e5d-8dff-34f3ee16c733 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:07:41.039610 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 631/1236] 10.222.0.20 () {56 vars in 1103 bytes} [Wed Aug 31 01:07:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:07:41.045486 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed43641-4d72-4335-a895-fa0a82796026 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:07:41.129385 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:07:41.145809 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed43641-4d72-4335-a895-fa0a82796026 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:07:41.177631 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546f6b83-0a45-4927-b5ca-8e18a38520ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:07:41.178266 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546f6b83-0a45-4927-b5ca-8e18a38520ac None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:07:41.178550 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546f6b83-0a45-4927-b5ca-8e18a38520ac None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:07:41.178828 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546f6b83-0a45-4927-b5ca-8e18a38520ac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:07:41.190367 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546f6b83-0a45-4927-b5ca-8e18a38520ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:07:41.191806 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546f6b83-0a45-4927-b5ca-8e18a38520ac None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:07:41.206424 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application [None req-546f6b83-0a45-4927-b5ca-8e18a38520ac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 01:07:41.206424 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:07:41.206424 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:07:41.206424 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:07:41.206424 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:07:41.206424 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:07:41.206424 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:07:41.206424 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:07:41.206424 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:07:41.206424 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:07:41.206424 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:07:41.206424 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:07:41.206424 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 01:07:41.206424 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 01:07:41.206424 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 01:07:41.206424 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 01:07:41.209945 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:07:41.209945 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:07:41.209945 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:07:41.209945 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:07:41.209945 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:07:41.209945 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:07:41.209945 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:07:41.209945 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application with Lock( Aug 31 01:07:41.209945 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:07:41.209945 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:07:41.209945 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:07:41.209945 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:07:41.209945 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:07:41.209945 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:07:41.209945 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:07:41.209945 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:07:41.209945 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 01:07:41.211130 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 01:07:41.211130 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 01:07:41.211130 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Aug 31 01:07:41.211130 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 606/1237] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 01:07:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:07:41.214066 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bdac13-58be-4e5d-8dff-34f3ee16c733 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:07:41.312636 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bdac13-58be-4e5d-8dff-34f3ee16c733 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:07:41.321051 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:07:41.345301 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1125c2-0f0a-483b-975a-6e40a14f45fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:07:41.345900 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1125c2-0f0a-483b-975a-6e40a14f45fb None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:07:41.346184 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1125c2-0f0a-483b-975a-6e40a14f45fb None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:07:41.346462 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1125c2-0f0a-483b-975a-6e40a14f45fb None admin] PATH_INFO: `/v3/domains` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:07:41.347089 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1125c2-0f0a-483b-975a-6e40a14f45fb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:07:41.347394 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1125c2-0f0a-483b-975a-6e40a14f45fb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:07:41.348775 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1125c2-0f0a-483b-975a-6e40a14f45fb None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:07:41.360445 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 632/1238] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 31 01:07:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:07:41.366312 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546f6b83-0a45-4927-b5ca-8e18a38520ac None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:07:41.465155 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546f6b83-0a45-4927-b5ca-8e18a38520ac None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:07:41.499119 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f96a8f-8057-4432-976c-3b47bffb6a56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:07:41.499743 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f96a8f-8057-4432-976c-3b47bffb6a56 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:07:41.500027 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f96a8f-8057-4432-976c-3b47bffb6a56 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:07:41.500302 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f96a8f-8057-4432-976c-3b47bffb6a56 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:07:41.502900 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:07:41.507901 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f96a8f-8057-4432-976c-3b47bffb6a56 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:07:41.509431 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f96a8f-8057-4432-976c-3b47bffb6a56 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:07:41.520331 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application [None req-29f96a8f-8057-4432-976c-3b47bffb6a56 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 31 01:07:41.520331 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:07:41.520331 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:07:41.520331 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:07:41.520331 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:07:41.520331 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:07:41.520331 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:07:41.520331 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:07:41.520331 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:07:41.520331 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:07:41.520331 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:07:41.520331 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:07:41.520331 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 01:07:41.520331 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 01:07:41.520331 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 01:07:41.520331 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 01:07:41.521746 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:07:41.521746 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:07:41.521746 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 01:07:41.521746 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 01:07:41.521746 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 01:07:41.521746 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 01:07:41.521746 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:07:41.521746 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:07:41.521746 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:07:41.521746 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:07:41.521746 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:07:41.521746 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application with Lock( Aug 31 01:07:41.521746 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:07:41.521746 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:07:41.521746 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:07:41.521746 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:07:41.521746 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:07:41.521746 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:07:41.522971 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:07:41.522971 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:07:41.522971 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 01:07:41.522971 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 01:07:41.522971 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 01:07:41.522971 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 01:07:41.522971 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 01:07:41.522971 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 01:07:41.522971 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 31 01:07:41.522971 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Aug 31 01:07:41.522971 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 607/1239] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 01:07:41 2022] GET /identity/v3/users/placement => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:07:41.527270 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1125c2-0f0a-483b-975a-6e40a14f45fb None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:07:41.625372 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1125c2-0f0a-483b-975a-6e40a14f45fb None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:07:41.657837 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc314ec6-5b59-4a95-acc4-61598307eb0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:07:41.658432 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc314ec6-5b59-4a95-acc4-61598307eb0c None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:07:41.658714 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc314ec6-5b59-4a95-acc4-61598307eb0c None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:07:41.658989 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc314ec6-5b59-4a95-acc4-61598307eb0c None admin] PATH_INFO: `/v3/users` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:07:41.659711 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc314ec6-5b59-4a95-acc4-61598307eb0c None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:07:41.660022 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc314ec6-5b59-4a95-acc4-61598307eb0c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:07:41.661477 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc314ec6-5b59-4a95-acc4-61598307eb0c None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:07:41.679124 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 633/1240] 10.222.0.20 () {56 vars in 1143 bytes} [Wed Aug 31 01:07:41 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:07:41.685606 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f96a8f-8057-4432-976c-3b47bffb6a56 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:07:41.782844 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f96a8f-8057-4432-976c-3b47bffb6a56 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:07:41.816975 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720dd210-e5e4-4453-8164-07ab67f8364a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:07:41.817742 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720dd210-e5e4-4453-8164-07ab67f8364a None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:07:41.818021 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720dd210-e5e4-4453-8164-07ab67f8364a None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:07:41.818291 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720dd210-e5e4-4453-8164-07ab67f8364a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:07:41.878722 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:07:41.885015 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720dd210-e5e4-4453-8164-07ab67f8364a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:07:41.893688 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720dd210-e5e4-4453-8164-07ab67f8364a None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:07:41.981936 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 608/1241] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Aug 31 01:07:41 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 01:07:41.987808 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc314ec6-5b59-4a95-acc4-61598307eb0c None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:07:42.015340 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:07:42.087993 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc314ec6-5b59-4a95-acc4-61598307eb0c None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:07:42.122641 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7e7639-b671-4bed-9bfd-39d1b13ddba4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:07:42.123268 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7e7639-b671-4bed-9bfd-39d1b13ddba4 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:07:42.123552 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7e7639-b671-4bed-9bfd-39d1b13ddba4 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:07:42.123824 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7e7639-b671-4bed-9bfd-39d1b13ddba4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:07:42.133617 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7e7639-b671-4bed-9bfd-39d1b13ddba4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:07:42.135043 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7e7639-b671-4bed-9bfd-39d1b13ddba4 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:07:42.150343 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application [None req-1c7e7639-b671-4bed-9bfd-39d1b13ddba4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 01:07:42.150343 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:07:42.150343 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:07:42.150343 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:07:42.150343 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:07:42.150343 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:07:42.150343 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:07:42.150343 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:07:42.150343 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:07:42.150343 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:07:42.150343 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:07:42.150343 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:07:42.150343 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 01:07:42.150343 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 01:07:42.150343 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 01:07:42.150343 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 01:07:42.151463 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:07:42.151463 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:07:42.151463 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:07:42.151463 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:07:42.151463 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:07:42.151463 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:07:42.151463 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:07:42.151463 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application with Lock( Aug 31 01:07:42.151463 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:07:42.151463 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:07:42.151463 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:07:42.151463 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:07:42.151463 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:07:42.151463 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:07:42.151463 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:07:42.151463 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:07:42.151463 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 01:07:42.152571 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 01:07:42.152571 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 01:07:42.152571 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Aug 31 01:07:42.152571 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 634/1242] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 01:07:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:07:42.157370 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720dd210-e5e4-4453-8164-07ab67f8364a None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:07:42.248069 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:07:42.253800 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720dd210-e5e4-4453-8164-07ab67f8364a None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:07:42.286919 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f6928d-f0b1-43e2-8ba4-b761cd07b85c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:07:42.287532 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f6928d-f0b1-43e2-8ba4-b761cd07b85c None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:07:42.287809 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f6928d-f0b1-43e2-8ba4-b761cd07b85c None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:07:42.288081 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f6928d-f0b1-43e2-8ba4-b761cd07b85c None admin] PATH_INFO: `/v3/domains` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:07:42.288709 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f6928d-f0b1-43e2-8ba4-b761cd07b85c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:07:42.289035 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f6928d-f0b1-43e2-8ba4-b761cd07b85c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:07:42.290909 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f6928d-f0b1-43e2-8ba4-b761cd07b85c None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:07:42.301816 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 609/1243] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 31 01:07:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:07:42.308459 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7e7639-b671-4bed-9bfd-39d1b13ddba4 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:07:42.385044 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:07:42.405057 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7e7639-b671-4bed-9bfd-39d1b13ddba4 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:07:42.437058 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1549d68-bf55-4f0a-b286-030089522580 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:07:42.437670 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1549d68-bf55-4f0a-b286-030089522580 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:07:42.437956 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1549d68-bf55-4f0a-b286-030089522580 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:07:42.438228 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1549d68-bf55-4f0a-b286-030089522580 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:07:42.444576 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1549d68-bf55-4f0a-b286-030089522580 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:07:42.446047 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1549d68-bf55-4f0a-b286-030089522580 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:07:42.457357 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application [None req-a1549d68-bf55-4f0a-b286-030089522580 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 01:07:42.457357 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:07:42.457357 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:07:42.457357 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:07:42.457357 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:07:42.457357 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:07:42.457357 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:07:42.457357 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:07:42.457357 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:07:42.457357 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:07:42.457357 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:07:42.457357 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:07:42.457357 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 01:07:42.457357 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 01:07:42.457357 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 01:07:42.458448 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 01:07:42.458448 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:07:42.458448 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:07:42.458448 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:07:42.458448 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:07:42.458448 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:07:42.458448 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:07:42.458448 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:07:42.458448 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application with Lock( Aug 31 01:07:42.458448 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:07:42.458448 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:07:42.458448 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:07:42.458448 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:07:42.458448 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:07:42.458448 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:07:42.458448 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:07:42.458448 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:07:42.459509 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 01:07:42.459509 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 01:07:42.459509 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 01:07:42.459509 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 01:07:42.459509 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 01:07:42.459509 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 01:07:42.459509 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 01:07:42.459509 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Aug 31 01:07:42.459509 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 635/1244] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Aug 31 01:07:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:07:42.464337 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f6928d-f0b1-43e2-8ba4-b761cd07b85c None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:07:42.564175 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f6928d-f0b1-43e2-8ba4-b761cd07b85c None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:07:42.596515 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b383975f-7fa8-40e5-b7f0-cd59175eccc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:07:42.597133 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b383975f-7fa8-40e5-b7f0-cd59175eccc8 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:07:42.597414 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b383975f-7fa8-40e5-b7f0-cd59175eccc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:07:42.597683 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b383975f-7fa8-40e5-b7f0-cd59175eccc8 None admin] PATH_INFO: `/v3/projects` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:07:42.598365 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b383975f-7fa8-40e5-b7f0-cd59175eccc8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:07:42.598667 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b383975f-7fa8-40e5-b7f0-cd59175eccc8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:07:42.600083 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b383975f-7fa8-40e5-b7f0-cd59175eccc8 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:07:42.612815 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 610/1245] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Aug 31 01:07:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:07:42.619655 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1549d68-bf55-4f0a-b286-030089522580 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:07:42.716001 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1549d68-bf55-4f0a-b286-030089522580 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:07:42.749050 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6342bf6c-2ea0-4353-8c8f-74f6367eabc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:07:42.749650 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6342bf6c-2ea0-4353-8c8f-74f6367eabc0 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:07:42.749931 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6342bf6c-2ea0-4353-8c8f-74f6367eabc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:07:42.750205 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6342bf6c-2ea0-4353-8c8f-74f6367eabc0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:07:42.750913 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6342bf6c-2ea0-4353-8c8f-74f6367eabc0 None admin] RBAC: Adding query filter params (role.id=7d00a4cfa13744a69604ac2cf1a5816e, scope.project.id=60b2abc15c2849ba8a1ed92117cc981b, user.id=dea0fe5b1d6b405cb47b95192580303b) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:07:42.751222 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6342bf6c-2ea0-4353-8c8f-74f6367eabc0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:07:42.752631 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6342bf6c-2ea0-4353-8c8f-74f6367eabc0 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:07:42.758278 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 636/1246] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Aug 31 01:07:42 2022] GET /identity/v3/role_assignments?user.id=dea0fe5b1d6b405cb47b95192580303b&scope.project.id=60b2abc15c2849ba8a1ed92117cc981b&role.id=7d00a4cfa13744a69604ac2cf1a5816e => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:07:44.697589 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00172d5b-0aa9-423a-a9d7-6033fa1998a3 None None] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:07:44.698227 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00172d5b-0aa9-423a-a9d7-6033fa1998a3 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:07:44.698227 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00172d5b-0aa9-423a-a9d7-6033fa1998a3 None None] PATH_INFO: `/` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:07:44.699534 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 611/1247] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:07:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:07:44.704841 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8a0d89-c1ca-438d-af42-40e44aea5bcd None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:07:44.705343 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8a0d89-c1ca-438d-af42-40e44aea5bcd None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:07:44.705535 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8a0d89-c1ca-438d-af42-40e44aea5bcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:07:44.711908 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:07:44.724496 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-2a8a0d89-c1ca-438d-af42-40e44aea5bcd None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:07:44.725730 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-2a8a0d89-c1ca-438d-af42-40e44aea5bcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:07:44.791704 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 637/1248] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:07:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 01:07:45.075375 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00172d5b-0aa9-423a-a9d7-6033fa1998a3 None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:07:45.094069 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:07:45.175467 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00172d5b-0aa9-423a-a9d7-6033fa1998a3 None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:07:45.210085 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37062d59-334a-4d7d-a0b5-3f947f63d913 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:07:45.210709 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37062d59-334a-4d7d-a0b5-3f947f63d913 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:07:45.210991 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37062d59-334a-4d7d-a0b5-3f947f63d913 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:07:45.211265 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37062d59-334a-4d7d-a0b5-3f947f63d913 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:07:45.223047 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37062d59-334a-4d7d-a0b5-3f947f63d913 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 01:07:45.227893 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application [None req-37062d59-334a-4d7d-a0b5-3f947f63d913 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 01:07:45.227893 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:07:45.227893 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:07:45.227893 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:07:45.227893 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:07:45.227893 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:07:45.227893 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:07:45.227893 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:07:45.227893 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:07:45.227893 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:07:45.227893 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:07:45.227893 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:07:45.227893 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 01:07:45.227893 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 01:07:45.227893 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 01:07:45.227893 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 01:07:45.229112 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 01:07:45.229112 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 01:07:45.229112 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 01:07:45.229112 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 01:07:45.229112 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:07:45.229112 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:07:45.229112 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:07:45.229112 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:07:45.229112 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:07:45.229112 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:07:45.229112 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:07:45.229112 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application with Lock( Aug 31 01:07:45.229112 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:07:45.229112 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:07:45.229112 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:07:45.229112 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:07:45.230036 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:07:45.230036 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:07:45.230036 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:07:45.230036 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:07:45.230036 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 01:07:45.230036 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 01:07:45.230036 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 01:07:45.230036 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 01:07:45.230036 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 01:07:45.230036 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 01:07:45.230036 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 01:07:45.230036 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Aug 31 01:07:45.230036 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 612/1249] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Aug 31 01:07:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:07:45.235104 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8a0d89-c1ca-438d-af42-40e44aea5bcd None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:07:45.333780 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8a0d89-c1ca-438d-af42-40e44aea5bcd None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:07:45.361590 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:07:45.366305 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea136f23-ceeb-4901-a25e-7f706997d451 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:07:45.366906 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea136f23-ceeb-4901-a25e-7f706997d451 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:07:45.367188 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea136f23-ceeb-4901-a25e-7f706997d451 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:07:45.367461 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea136f23-ceeb-4901-a25e-7f706997d451 None admin] PATH_INFO: `/v3/roles` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:07:45.368117 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea136f23-ceeb-4901-a25e-7f706997d451 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:07:45.368966 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea136f23-ceeb-4901-a25e-7f706997d451 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:07:45.370328 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea136f23-ceeb-4901-a25e-7f706997d451 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:07:45.378110 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 638/1250] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Aug 31 01:07:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:07:45.384773 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37062d59-334a-4d7d-a0b5-3f947f63d913 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:07:45.461834 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:07:45.484636 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37062d59-334a-4d7d-a0b5-3f947f63d913 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:07:45.518041 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07054eb4-fef7-4e39-8358-edc52b0c6526 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:07:45.518679 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07054eb4-fef7-4e39-8358-edc52b0c6526 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:07:45.518956 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07054eb4-fef7-4e39-8358-edc52b0c6526 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:07:45.519235 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07054eb4-fef7-4e39-8358-edc52b0c6526 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:07:45.529790 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07054eb4-fef7-4e39-8358-edc52b0c6526 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:07:45.531259 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07054eb4-fef7-4e39-8358-edc52b0c6526 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:07:45.546688 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application [None req-07054eb4-fef7-4e39-8358-edc52b0c6526 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 01:07:45.546688 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:07:45.546688 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:07:45.546688 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:07:45.546688 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:07:45.546688 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:07:45.546688 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:07:45.546688 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:07:45.546688 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:07:45.546688 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:07:45.546688 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:07:45.546688 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:07:45.546688 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 01:07:45.546688 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 01:07:45.546688 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 01:07:45.546688 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 01:07:45.550543 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:07:45.550543 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:07:45.550543 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:07:45.550543 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:07:45.550543 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:07:45.550543 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:07:45.550543 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:07:45.550543 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application with Lock( Aug 31 01:07:45.550543 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:07:45.550543 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:07:45.550543 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:07:45.550543 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:07:45.550543 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:07:45.550543 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:07:45.550543 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:07:45.550543 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:07:45.550543 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 01:07:45.551643 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 01:07:45.551643 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 01:07:45.551643 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Aug 31 01:07:45.551643 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 613/1251] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 01:07:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:07:45.553591 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea136f23-ceeb-4901-a25e-7f706997d451 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:07:45.656256 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea136f23-ceeb-4901-a25e-7f706997d451 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:07:45.689352 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4a5a46-74d1-40b6-9935-6bf1f27a67b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:07:45.690002 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4a5a46-74d1-40b6-9935-6bf1f27a67b9 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:07:45.690328 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4a5a46-74d1-40b6-9935-6bf1f27a67b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:07:45.690598 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4a5a46-74d1-40b6-9935-6bf1f27a67b9 None admin] PATH_INFO: `/v3/domains` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:07:45.691232 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4a5a46-74d1-40b6-9935-6bf1f27a67b9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:07:45.691533 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4a5a46-74d1-40b6-9935-6bf1f27a67b9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:07:45.692933 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4a5a46-74d1-40b6-9935-6bf1f27a67b9 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:07:45.704817 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 639/1252] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 31 01:07:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:07:45.710673 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07054eb4-fef7-4e39-8358-edc52b0c6526 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:07:45.810984 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07054eb4-fef7-4e39-8358-edc52b0c6526 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:07:45.836162 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:07:45.843710 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76826f77-2ff1-4eed-bf1c-5654a9f770e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:07:45.844362 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76826f77-2ff1-4eed-bf1c-5654a9f770e4 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:07:45.844646 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76826f77-2ff1-4eed-bf1c-5654a9f770e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:07:45.844952 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76826f77-2ff1-4eed-bf1c-5654a9f770e4 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:07:45.852078 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76826f77-2ff1-4eed-bf1c-5654a9f770e4 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:07:45.854076 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76826f77-2ff1-4eed-bf1c-5654a9f770e4 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:07:45.864916 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application [None req-76826f77-2ff1-4eed-bf1c-5654a9f770e4 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 31 01:07:45.864916 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:07:45.864916 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:07:45.864916 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:07:45.864916 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:07:45.864916 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:07:45.864916 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:07:45.864916 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:07:45.864916 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:07:45.864916 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:07:45.864916 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:07:45.864916 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:07:45.864916 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 01:07:45.864916 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 01:07:45.864916 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 01:07:45.864916 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 01:07:45.867342 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:07:45.867342 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:07:45.867342 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 01:07:45.867342 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 01:07:45.867342 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 01:07:45.867342 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 01:07:45.867342 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:07:45.867342 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:07:45.867342 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:07:45.867342 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:07:45.867342 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:07:45.867342 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application with Lock( Aug 31 01:07:45.867342 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:07:45.867342 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:07:45.867342 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:07:45.867342 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:07:45.867342 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:07:45.867342 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:07:45.869202 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:07:45.869202 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:07:45.869202 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 01:07:45.869202 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 01:07:45.869202 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 01:07:45.869202 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 01:07:45.869202 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 01:07:45.869202 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 01:07:45.869202 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 31 01:07:45.869202 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Aug 31 01:07:45.869202 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 614/1253] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 01:07:45 2022] GET /identity/v3/users/placement => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:07:45.872373 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4a5a46-74d1-40b6-9935-6bf1f27a67b9 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:07:45.930298 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:07:45.973735 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4a5a46-74d1-40b6-9935-6bf1f27a67b9 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:07:46.008041 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf35cfe3-3fad-4214-bab8-b72622beb8ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:07:46.008645 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf35cfe3-3fad-4214-bab8-b72622beb8ed None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:07:46.009095 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf35cfe3-3fad-4214-bab8-b72622beb8ed None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:07:46.009436 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf35cfe3-3fad-4214-bab8-b72622beb8ed None admin] PATH_INFO: `/v3/users` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:07:46.010160 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf35cfe3-3fad-4214-bab8-b72622beb8ed None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:07:46.010471 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf35cfe3-3fad-4214-bab8-b72622beb8ed None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:07:46.011965 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf35cfe3-3fad-4214-bab8-b72622beb8ed None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:07:46.029287 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 640/1254] 10.222.0.20 () {56 vars in 1143 bytes} [Wed Aug 31 01:07:45 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:07:46.035118 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76826f77-2ff1-4eed-bf1c-5654a9f770e4 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:07:46.135128 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76826f77-2ff1-4eed-bf1c-5654a9f770e4 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:07:46.168948 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3276d46d-0375-4953-87d8-a90b24373569 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:07:46.169548 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3276d46d-0375-4953-87d8-a90b24373569 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:07:46.169882 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3276d46d-0375-4953-87d8-a90b24373569 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:07:46.170161 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3276d46d-0375-4953-87d8-a90b24373569 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:07:46.227080 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:07:46.239117 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3276d46d-0375-4953-87d8-a90b24373569 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:07:46.240629 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3276d46d-0375-4953-87d8-a90b24373569 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:07:46.338854 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 615/1255] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Aug 31 01:07:46 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 01:07:46.345113 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf35cfe3-3fad-4214-bab8-b72622beb8ed None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:07:46.442797 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf35cfe3-3fad-4214-bab8-b72622beb8ed None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:07:46.456391 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:07:46.477031 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53703c0f-be49-465c-89bc-62cd4f69f8c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:07:46.477706 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53703c0f-be49-465c-89bc-62cd4f69f8c1 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:07:46.477933 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53703c0f-be49-465c-89bc-62cd4f69f8c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:07:46.478206 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53703c0f-be49-465c-89bc-62cd4f69f8c1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:07:46.487951 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53703c0f-be49-465c-89bc-62cd4f69f8c1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:07:46.489430 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53703c0f-be49-465c-89bc-62cd4f69f8c1 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:07:46.503589 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application [None req-53703c0f-be49-465c-89bc-62cd4f69f8c1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 01:07:46.503589 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:07:46.503589 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:07:46.503589 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:07:46.503589 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:07:46.503589 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:07:46.503589 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:07:46.503589 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:07:46.503589 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:07:46.503589 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:07:46.503589 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:07:46.503589 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:07:46.503589 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 01:07:46.503589 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 01:07:46.503589 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 01:07:46.503589 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 01:07:46.504713 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:07:46.504713 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:07:46.504713 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:07:46.504713 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:07:46.504713 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:07:46.504713 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:07:46.504713 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:07:46.504713 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application with Lock( Aug 31 01:07:46.504713 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:07:46.504713 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:07:46.504713 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:07:46.504713 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:07:46.504713 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:07:46.504713 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:07:46.504713 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:07:46.504713 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:07:46.504713 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 01:07:46.505684 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 01:07:46.505684 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 01:07:46.505684 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Aug 31 01:07:46.505684 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 641/1256] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 01:07:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:07:46.511309 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3276d46d-0375-4953-87d8-a90b24373569 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:07:46.597173 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:07:46.608829 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3276d46d-0375-4953-87d8-a90b24373569 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:07:46.642343 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1536b8ad-a17a-481a-b5d7-6b4dcb45570d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:07:46.642935 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1536b8ad-a17a-481a-b5d7-6b4dcb45570d None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:07:46.643216 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1536b8ad-a17a-481a-b5d7-6b4dcb45570d None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:07:46.643488 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1536b8ad-a17a-481a-b5d7-6b4dcb45570d None admin] PATH_INFO: `/v3/domains` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:07:46.644119 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1536b8ad-a17a-481a-b5d7-6b4dcb45570d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:07:46.644423 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1536b8ad-a17a-481a-b5d7-6b4dcb45570d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:07:46.645815 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1536b8ad-a17a-481a-b5d7-6b4dcb45570d None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:07:46.656644 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 616/1257] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 31 01:07:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:07:46.662445 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53703c0f-be49-465c-89bc-62cd4f69f8c1 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:07:46.761757 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53703c0f-be49-465c-89bc-62cd4f69f8c1 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:07:46.793756 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8cedc1-d598-488e-a037-2d98ca0a70e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:07:46.794434 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8cedc1-d598-488e-a037-2d98ca0a70e3 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:07:46.794699 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8cedc1-d598-488e-a037-2d98ca0a70e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:07:46.794974 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8cedc1-d598-488e-a037-2d98ca0a70e3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:07:46.801317 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8cedc1-d598-488e-a037-2d98ca0a70e3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:07:46.802831 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8cedc1-d598-488e-a037-2d98ca0a70e3 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:07:46.814310 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application [None req-ba8cedc1-d598-488e-a037-2d98ca0a70e3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 01:07:46.814310 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:07:46.814310 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:07:46.814310 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:07:46.814310 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:07:46.814310 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:07:46.814310 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:07:46.814310 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:07:46.814310 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:07:46.814310 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:07:46.814310 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:07:46.814310 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:07:46.814310 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 01:07:46.814310 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 01:07:46.814310 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 01:07:46.815347 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 01:07:46.815347 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:07:46.815347 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:07:46.815347 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:07:46.815347 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:07:46.815347 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:07:46.815347 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:07:46.815347 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:07:46.815347 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application with Lock( Aug 31 01:07:46.815347 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:07:46.815347 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:07:46.815347 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:07:46.815347 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:07:46.815347 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:07:46.815347 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:07:46.815347 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:07:46.815347 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:07:46.821102 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 01:07:46.821102 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 01:07:46.821102 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 01:07:46.821102 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 01:07:46.821102 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 01:07:46.821102 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 01:07:46.821102 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 01:07:46.821102 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Aug 31 01:07:46.821102 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 642/1258] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Aug 31 01:07:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:07:46.822281 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1536b8ad-a17a-481a-b5d7-6b4dcb45570d None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:07:46.921397 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1536b8ad-a17a-481a-b5d7-6b4dcb45570d None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:07:46.955356 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3156023-fc08-42c3-b7f7-2104fb903a7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:07:46.955963 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3156023-fc08-42c3-b7f7-2104fb903a7c None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:07:46.956247 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3156023-fc08-42c3-b7f7-2104fb903a7c None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:07:46.956527 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3156023-fc08-42c3-b7f7-2104fb903a7c None admin] PATH_INFO: `/v3/projects` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:07:46.957250 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3156023-fc08-42c3-b7f7-2104fb903a7c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:07:46.957562 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3156023-fc08-42c3-b7f7-2104fb903a7c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:07:46.958997 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3156023-fc08-42c3-b7f7-2104fb903a7c None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:07:46.970901 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 617/1259] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Aug 31 01:07:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:07:46.977108 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8cedc1-d598-488e-a037-2d98ca0a70e3 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:07:47.003755 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:07:47.076939 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8cedc1-d598-488e-a037-2d98ca0a70e3 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:07:47.109779 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9829196b-d3fa-4efb-ae5f-2f173d7f01dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:07:47.110378 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9829196b-d3fa-4efb-ae5f-2f173d7f01dc None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:07:47.110657 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9829196b-d3fa-4efb-ae5f-2f173d7f01dc None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:07:47.110929 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9829196b-d3fa-4efb-ae5f-2f173d7f01dc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:07:47.111645 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9829196b-d3fa-4efb-ae5f-2f173d7f01dc None admin] RBAC: Adding query filter params (role.id=d009411496b8452b88e77ffb8db015a7, scope.project.id=60b2abc15c2849ba8a1ed92117cc981b, user.id=dea0fe5b1d6b405cb47b95192580303b) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:07:47.111955 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9829196b-d3fa-4efb-ae5f-2f173d7f01dc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:07:47.113435 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9829196b-d3fa-4efb-ae5f-2f173d7f01dc None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:07:47.119198 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 643/1260] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Aug 31 01:07:46 2022] GET /identity/v3/role_assignments?user.id=dea0fe5b1d6b405cb47b95192580303b&scope.project.id=60b2abc15c2849ba8a1ed92117cc981b&role.id=d009411496b8452b88e77ffb8db015a7 => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:07:48.990112 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8c59f4-514a-4625-9d67-4b4814fb9ad0 None None] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:07:48.990623 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8c59f4-514a-4625-9d67-4b4814fb9ad0 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:07:48.990713 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8c59f4-514a-4625-9d67-4b4814fb9ad0 None None] PATH_INFO: `/` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:07:48.991985 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 618/1261] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:07:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:07:48.997023 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec62480-2bc0-42b4-9807-167a7c478ba7 None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:07:48.997429 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec62480-2bc0-42b4-9807-167a7c478ba7 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:07:48.997577 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec62480-2bc0-42b4-9807-167a7c478ba7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:07:49.015269 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-5ec62480-2bc0-42b4-9807-167a7c478ba7 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:07:49.016455 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-5ec62480-2bc0-42b4-9807-167a7c478ba7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:07:49.081315 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 644/1262] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:07:48 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 01:07:49.350716 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8c59f4-514a-4625-9d67-4b4814fb9ad0 None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:07:49.361527 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:07:49.453719 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8c59f4-514a-4625-9d67-4b4814fb9ad0 None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:07:49.487464 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73a5f66-306c-4b86-bcdc-e561e5e28680 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:07:49.488085 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73a5f66-306c-4b86-bcdc-e561e5e28680 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:07:49.488363 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73a5f66-306c-4b86-bcdc-e561e5e28680 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:07:49.488634 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73a5f66-306c-4b86-bcdc-e561e5e28680 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:07:49.498645 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73a5f66-306c-4b86-bcdc-e561e5e28680 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 01:07:49.503485 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application [None req-d73a5f66-306c-4b86-bcdc-e561e5e28680 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 01:07:49.503485 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:07:49.503485 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:07:49.503485 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:07:49.503485 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:07:49.503485 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:07:49.503485 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:07:49.503485 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:07:49.503485 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:07:49.503485 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:07:49.503485 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:07:49.503485 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:07:49.503485 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 01:07:49.503485 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 01:07:49.503485 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 01:07:49.503485 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 01:07:49.504450 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 01:07:49.504450 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 01:07:49.504450 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 01:07:49.504450 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 01:07:49.504450 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:07:49.504450 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:07:49.504450 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:07:49.504450 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:07:49.504450 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:07:49.504450 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:07:49.504450 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:07:49.504450 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application with Lock( Aug 31 01:07:49.504450 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:07:49.504450 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:07:49.504450 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:07:49.504450 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:07:49.505348 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:07:49.505348 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:07:49.505348 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:07:49.505348 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:07:49.505348 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 01:07:49.505348 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 01:07:49.505348 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 01:07:49.505348 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 01:07:49.505348 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 01:07:49.505348 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 01:07:49.505348 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 01:07:49.505348 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Aug 31 01:07:49.505348 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 619/1263] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Aug 31 01:07:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:07:49.511358 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec62480-2bc0-42b4-9807-167a7c478ba7 None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:07:49.530221 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:07:49.613313 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec62480-2bc0-42b4-9807-167a7c478ba7 None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:07:49.646394 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d73437-ce9b-489c-a867-dfa1b3c098b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:07:49.646986 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d73437-ce9b-489c-a867-dfa1b3c098b4 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:07:49.647263 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d73437-ce9b-489c-a867-dfa1b3c098b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:07:49.647532 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d73437-ce9b-489c-a867-dfa1b3c098b4 None admin] PATH_INFO: `/v3/roles` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:07:49.648182 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d73437-ce9b-489c-a867-dfa1b3c098b4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:07:49.648479 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d73437-ce9b-489c-a867-dfa1b3c098b4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:07:49.649898 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d73437-ce9b-489c-a867-dfa1b3c098b4 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:07:49.657570 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 645/1264] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Aug 31 01:07:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:07:49.663450 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73a5f66-306c-4b86-bcdc-e561e5e28680 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:07:49.730157 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:07:49.762717 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73a5f66-306c-4b86-bcdc-e561e5e28680 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:07:49.795807 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad14e3e5-62a8-45fa-a543-b3de8cfdd328 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:07:49.796430 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad14e3e5-62a8-45fa-a543-b3de8cfdd328 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:07:49.796755 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad14e3e5-62a8-45fa-a543-b3de8cfdd328 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:07:49.797019 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad14e3e5-62a8-45fa-a543-b3de8cfdd328 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:07:49.807352 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad14e3e5-62a8-45fa-a543-b3de8cfdd328 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:07:49.808819 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad14e3e5-62a8-45fa-a543-b3de8cfdd328 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:07:49.823811 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application [None req-ad14e3e5-62a8-45fa-a543-b3de8cfdd328 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 01:07:49.823811 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:07:49.823811 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:07:49.823811 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:07:49.823811 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:07:49.823811 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:07:49.823811 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:07:49.823811 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:07:49.823811 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:07:49.823811 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:07:49.823811 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:07:49.823811 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:07:49.823811 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 01:07:49.823811 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 01:07:49.823811 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 01:07:49.823811 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 01:07:49.825269 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:07:49.825269 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:07:49.825269 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:07:49.825269 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:07:49.825269 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:07:49.825269 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:07:49.825269 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:07:49.825269 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application with Lock( Aug 31 01:07:49.825269 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:07:49.825269 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:07:49.825269 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:07:49.825269 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:07:49.825269 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:07:49.825269 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:07:49.825269 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:07:49.825269 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:07:49.825269 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 01:07:49.826391 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 01:07:49.826391 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 01:07:49.826391 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Aug 31 01:07:49.826391 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 620/1265] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 01:07:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:07:49.831137 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d73437-ce9b-489c-a867-dfa1b3c098b4 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:07:49.920219 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:07:49.938795 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d73437-ce9b-489c-a867-dfa1b3c098b4 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:07:49.974920 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddd8e7c-f2a9-480b-8251-928aa5165651 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:07:49.975523 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddd8e7c-f2a9-480b-8251-928aa5165651 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:07:49.975805 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddd8e7c-f2a9-480b-8251-928aa5165651 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:07:49.976080 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddd8e7c-f2a9-480b-8251-928aa5165651 None admin] PATH_INFO: `/v3/domains` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:07:49.976712 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddd8e7c-f2a9-480b-8251-928aa5165651 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:07:49.977045 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddd8e7c-f2a9-480b-8251-928aa5165651 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:07:49.978918 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddd8e7c-f2a9-480b-8251-928aa5165651 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:07:49.990399 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 646/1266] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 31 01:07:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:07:49.996881 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad14e3e5-62a8-45fa-a543-b3de8cfdd328 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:07:50.099144 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad14e3e5-62a8-45fa-a543-b3de8cfdd328 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:07:50.119118 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:07:50.132574 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f546c5-6e24-4e07-8169-276f8c1c23cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:07:50.133235 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f546c5-6e24-4e07-8169-276f8c1c23cc None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:07:50.133522 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f546c5-6e24-4e07-8169-276f8c1c23cc None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:07:50.133802 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f546c5-6e24-4e07-8169-276f8c1c23cc None admin] PATH_INFO: `/v3/users/placement` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:07:50.141013 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f546c5-6e24-4e07-8169-276f8c1c23cc None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:07:50.142453 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f546c5-6e24-4e07-8169-276f8c1c23cc None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:07:50.154633 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application [None req-24f546c5-6e24-4e07-8169-276f8c1c23cc None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 31 01:07:50.154633 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:07:50.154633 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:07:50.154633 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:07:50.154633 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:07:50.154633 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:07:50.154633 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:07:50.154633 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:07:50.154633 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:07:50.154633 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:07:50.154633 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:07:50.154633 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:07:50.154633 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 01:07:50.154633 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 01:07:50.154633 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 01:07:50.154633 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 01:07:50.155719 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:07:50.155719 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:07:50.155719 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 01:07:50.155719 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 01:07:50.155719 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 01:07:50.155719 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 01:07:50.155719 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:07:50.155719 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:07:50.155719 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:07:50.155719 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:07:50.155719 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:07:50.155719 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application with Lock( Aug 31 01:07:50.155719 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:07:50.155719 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:07:50.155719 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:07:50.155719 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:07:50.155719 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:07:50.155719 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:07:50.156676 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:07:50.156676 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:07:50.156676 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 01:07:50.156676 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 01:07:50.156676 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 01:07:50.156676 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 01:07:50.156676 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 01:07:50.156676 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 01:07:50.156676 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 31 01:07:50.156676 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Aug 31 01:07:50.156676 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 621/1267] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 01:07:49 2022] GET /identity/v3/users/placement => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:07:50.160490 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddd8e7c-f2a9-480b-8251-928aa5165651 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:07:50.261926 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddd8e7c-f2a9-480b-8251-928aa5165651 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:07:50.296293 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8981960e-3b5e-4a95-a074-0a14f7946ae5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:07:50.296929 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8981960e-3b5e-4a95-a074-0a14f7946ae5 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:07:50.297216 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8981960e-3b5e-4a95-a074-0a14f7946ae5 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:07:50.297494 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8981960e-3b5e-4a95-a074-0a14f7946ae5 None admin] PATH_INFO: `/v3/users` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:07:50.298373 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8981960e-3b5e-4a95-a074-0a14f7946ae5 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:07:50.298688 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8981960e-3b5e-4a95-a074-0a14f7946ae5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:07:50.300109 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8981960e-3b5e-4a95-a074-0a14f7946ae5 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:07:50.309289 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:07:50.317530 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 647/1268] 10.222.0.20 () {56 vars in 1143 bytes} [Wed Aug 31 01:07:50 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:07:50.323435 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f546c5-6e24-4e07-8169-276f8c1c23cc None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:07:50.424344 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f546c5-6e24-4e07-8169-276f8c1c23cc None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:07:50.457121 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9742e58d-d18a-4e60-ac78-701d223363bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:07:50.458498 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9742e58d-d18a-4e60-ac78-701d223363bc None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:07:50.458498 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9742e58d-d18a-4e60-ac78-701d223363bc None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:07:50.458498 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9742e58d-d18a-4e60-ac78-701d223363bc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:07:50.469409 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9742e58d-d18a-4e60-ac78-701d223363bc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:07:50.471050 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9742e58d-d18a-4e60-ac78-701d223363bc None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:07:50.486072 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application [None req-9742e58d-d18a-4e60-ac78-701d223363bc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 01:07:50.486072 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:07:50.486072 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:07:50.486072 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:07:50.486072 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:07:50.486072 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:07:50.486072 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:07:50.486072 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:07:50.486072 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:07:50.486072 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:07:50.486072 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:07:50.486072 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:07:50.486072 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 01:07:50.486072 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 01:07:50.486072 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 01:07:50.486072 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 01:07:50.487245 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:07:50.487245 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:07:50.487245 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:07:50.487245 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:07:50.487245 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:07:50.487245 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:07:50.487245 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:07:50.487245 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application with Lock( Aug 31 01:07:50.487245 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:07:50.487245 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:07:50.487245 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:07:50.487245 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:07:50.487245 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:07:50.487245 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:07:50.487245 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:07:50.487245 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:07:50.487245 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 01:07:50.488389 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 01:07:50.488389 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 01:07:50.488389 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Aug 31 01:07:50.488389 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 622/1269] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 01:07:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:07:50.493487 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8981960e-3b5e-4a95-a074-0a14f7946ae5 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:07:50.593996 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8981960e-3b5e-4a95-a074-0a14f7946ae5 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:07:50.628614 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0032df8-b38b-4d41-9156-8373d1ade179 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:07:50.629291 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0032df8-b38b-4d41-9156-8373d1ade179 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:07:50.629577 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0032df8-b38b-4d41-9156-8373d1ade179 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:07:50.629853 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0032df8-b38b-4d41-9156-8373d1ade179 None admin] PATH_INFO: `/v3/domains` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:07:50.630489 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0032df8-b38b-4d41-9156-8373d1ade179 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:07:50.630795 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0032df8-b38b-4d41-9156-8373d1ade179 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:07:50.632165 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0032df8-b38b-4d41-9156-8373d1ade179 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:07:50.643166 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 648/1270] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 31 01:07:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:07:50.649067 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9742e58d-d18a-4e60-ac78-701d223363bc None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:07:50.681073 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:07:50.748901 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9742e58d-d18a-4e60-ac78-701d223363bc None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:07:50.782093 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061380bb-4cd7-489c-9712-246561c520b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:07:50.782772 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061380bb-4cd7-489c-9712-246561c520b2 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:07:50.783060 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061380bb-4cd7-489c-9712-246561c520b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:07:50.783347 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061380bb-4cd7-489c-9712-246561c520b2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:07:50.789801 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061380bb-4cd7-489c-9712-246561c520b2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:07:50.795552 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061380bb-4cd7-489c-9712-246561c520b2 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:07:50.802288 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application [None req-061380bb-4cd7-489c-9712-246561c520b2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 01:07:50.802288 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:07:50.802288 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:07:50.802288 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:07:50.802288 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:07:50.802288 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:07:50.802288 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:07:50.802288 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:07:50.802288 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:07:50.802288 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:07:50.802288 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:07:50.802288 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:07:50.802288 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 01:07:50.802288 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 01:07:50.802288 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 01:07:50.803398 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 01:07:50.803398 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:07:50.803398 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:07:50.803398 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:07:50.803398 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:07:50.803398 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:07:50.803398 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:07:50.803398 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:07:50.803398 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application with Lock( Aug 31 01:07:50.803398 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:07:50.803398 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:07:50.803398 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:07:50.803398 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:07:50.803398 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:07:50.803398 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:07:50.803398 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:07:50.803398 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:07:50.804712 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 01:07:50.804712 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 01:07:50.804712 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 01:07:50.804712 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 01:07:50.804712 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 01:07:50.804712 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 01:07:50.804712 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 01:07:50.804712 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Aug 31 01:07:50.804712 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 623/1271] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Aug 31 01:07:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:07:50.809297 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0032df8-b38b-4d41-9156-8373d1ade179 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:07:50.866541 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:07:50.910238 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0032df8-b38b-4d41-9156-8373d1ade179 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:07:50.942866 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f89448-bd88-4ec8-a72c-18eed23f4731 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:07:50.943413 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f89448-bd88-4ec8-a72c-18eed23f4731 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:07:50.943691 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f89448-bd88-4ec8-a72c-18eed23f4731 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:07:50.943961 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f89448-bd88-4ec8-a72c-18eed23f4731 None admin] PATH_INFO: `/v3/projects` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:07:50.944636 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f89448-bd88-4ec8-a72c-18eed23f4731 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:07:50.944968 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f89448-bd88-4ec8-a72c-18eed23f4731 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:07:50.946382 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f89448-bd88-4ec8-a72c-18eed23f4731 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:07:50.958672 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 649/1272] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Aug 31 01:07:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:07:50.965483 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061380bb-4cd7-489c-9712-246561c520b2 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:07:51.056222 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:07:51.064518 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061380bb-4cd7-489c-9712-246561c520b2 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:07:51.098241 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e036542-5dfc-47d4-8384-15171fb4a8be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:07:51.098891 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e036542-5dfc-47d4-8384-15171fb4a8be None admin] REQUEST_METHOD: `PUT` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:07:51.099175 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e036542-5dfc-47d4-8384-15171fb4a8be None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:07:51.099455 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e036542-5dfc-47d4-8384-15171fb4a8be None admin] PATH_INFO: `/v3/projects/60b2abc15c2849ba8a1ed92117cc981b/users/dea0fe5b1d6b405cb47b95192580303b/roles/d009411496b8452b88e77ffb8db015a7` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:07:51.101415 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e036542-5dfc-47d4-8384-15171fb4a8be None admin] RBAC: Authorizing `identity:create_grant(project_id=60b2abc15c2849ba8a1ed92117cc981b, user_id=dea0fe5b1d6b405cb47b95192580303b, role_id=d009411496b8452b88e77ffb8db015a7)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:07:51.102945 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e036542-5dfc-47d4-8384-15171fb4a8be None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:07:51.304305 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 624/1273] 10.222.0.20 () {58 vars in 1439 bytes} [Wed Aug 31 01:07:50 2022] PUT /identity/v3/projects/60b2abc15c2849ba8a1ed92117cc981b/users/dea0fe5b1d6b405cb47b95192580303b/roles/d009411496b8452b88e77ffb8db015a7 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:07:53.167947 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b411f15a-82d3-46f5-b34d-d3afa81b5ec9 None None] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:07:53.168496 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b411f15a-82d3-46f5-b34d-d3afa81b5ec9 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:07:53.168496 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b411f15a-82d3-46f5-b34d-d3afa81b5ec9 None None] PATH_INFO: `/` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:07:53.169870 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 650/1274] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:07:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:07:53.175099 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228046c3-a900-4976-ad52-80935838f19e None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:07:53.175520 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228046c3-a900-4976-ad52-80935838f19e None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:07:53.175647 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228046c3-a900-4976-ad52-80935838f19e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:07:53.193964 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-228046c3-a900-4976-ad52-80935838f19e None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:07:53.195166 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-228046c3-a900-4976-ad52-80935838f19e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:07:53.259722 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 625/1275] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:07:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 01:07:53.530868 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b411f15a-82d3-46f5-b34d-d3afa81b5ec9 None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:07:53.632821 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b411f15a-82d3-46f5-b34d-d3afa81b5ec9 None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:07:53.649318 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:07:53.666326 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b09bf2c-8c5b-49ce-841d-49686297de42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:07:53.666952 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b09bf2c-8c5b-49ce-841d-49686297de42 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:07:53.667234 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b09bf2c-8c5b-49ce-841d-49686297de42 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:07:53.667510 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b09bf2c-8c5b-49ce-841d-49686297de42 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:07:53.677147 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b09bf2c-8c5b-49ce-841d-49686297de42 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 01:07:53.682151 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application [None req-9b09bf2c-8c5b-49ce-841d-49686297de42 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 01:07:53.682151 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:07:53.682151 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:07:53.682151 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:07:53.682151 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:07:53.682151 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:07:53.682151 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:07:53.682151 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:07:53.682151 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:07:53.682151 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:07:53.682151 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:07:53.682151 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:07:53.682151 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 01:07:53.682151 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 01:07:53.682151 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 01:07:53.682151 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 01:07:53.683230 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 01:07:53.683230 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 01:07:53.683230 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 01:07:53.683230 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 01:07:53.683230 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:07:53.683230 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:07:53.683230 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:07:53.683230 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:07:53.683230 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:07:53.683230 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:07:53.683230 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:07:53.683230 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application with Lock( Aug 31 01:07:53.683230 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:07:53.683230 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:07:53.683230 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:07:53.683230 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:07:53.684102 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:07:53.684102 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:07:53.684102 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:07:53.684102 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:07:53.684102 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 01:07:53.684102 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 01:07:53.684102 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 01:07:53.684102 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 01:07:53.684102 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 01:07:53.684102 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 01:07:53.684102 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 01:07:53.684102 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Aug 31 01:07:53.684102 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 651/1276] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Aug 31 01:07:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:07:53.689559 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228046c3-a900-4976-ad52-80935838f19e None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:07:53.768179 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:07:53.788671 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228046c3-a900-4976-ad52-80935838f19e None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:07:53.819143 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538a1a34-9973-41d5-ad82-5f332a4df7ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:07:53.820339 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538a1a34-9973-41d5-ad82-5f332a4df7ed None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:07:53.820339 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538a1a34-9973-41d5-ad82-5f332a4df7ed None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:07:53.820339 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538a1a34-9973-41d5-ad82-5f332a4df7ed None admin] PATH_INFO: `/v3/roles` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:07:53.821027 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538a1a34-9973-41d5-ad82-5f332a4df7ed None admin] RBAC: Adding query filter params (name=admin) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:07:53.821353 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538a1a34-9973-41d5-ad82-5f332a4df7ed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:07:53.823213 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538a1a34-9973-41d5-ad82-5f332a4df7ed None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:07:53.831336 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 626/1277] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Aug 31 01:07:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:07:53.837016 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b09bf2c-8c5b-49ce-841d-49686297de42 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:07:53.935131 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b09bf2c-8c5b-49ce-841d-49686297de42 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:07:53.970656 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0231533c-5231-4528-a93a-aa184f3cd233 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:07:53.971275 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0231533c-5231-4528-a93a-aa184f3cd233 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:07:53.971557 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0231533c-5231-4528-a93a-aa184f3cd233 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:07:53.971829 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0231533c-5231-4528-a93a-aa184f3cd233 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:07:53.981400 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0231533c-5231-4528-a93a-aa184f3cd233 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:07:53.982832 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0231533c-5231-4528-a93a-aa184f3cd233 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:07:53.996864 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application [None req-0231533c-5231-4528-a93a-aa184f3cd233 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 01:07:53.996864 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:07:53.996864 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:07:53.996864 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:07:53.996864 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:07:53.996864 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:07:53.996864 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:07:53.996864 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:07:53.996864 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:07:53.996864 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:07:53.996864 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:07:53.996864 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:07:53.996864 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 01:07:53.996864 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 01:07:53.996864 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 01:07:53.996864 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 01:07:53.997995 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:07:53.997995 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:07:53.997995 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:07:53.997995 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:07:53.997995 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:07:53.997995 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:07:53.997995 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:07:53.997995 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application with Lock( Aug 31 01:07:53.997995 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:07:53.997995 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:07:53.997995 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:07:53.997995 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:07:53.997995 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:07:53.997995 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:07:53.997995 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:07:53.997995 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:07:53.997995 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 01:07:53.998980 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 01:07:53.998980 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 01:07:53.998980 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Aug 31 01:07:53.998980 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 652/1278] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 01:07:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:07:54.003805 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538a1a34-9973-41d5-ad82-5f332a4df7ed None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:07:54.103448 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538a1a34-9973-41d5-ad82-5f332a4df7ed None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:07:54.136531 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700af215-4b00-4b44-a1bd-1a6c3603398f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:07:54.140757 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700af215-4b00-4b44-a1bd-1a6c3603398f None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:07:54.140757 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700af215-4b00-4b44-a1bd-1a6c3603398f None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:07:54.140757 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700af215-4b00-4b44-a1bd-1a6c3603398f None admin] PATH_INFO: `/v3/domains` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:07:54.140757 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700af215-4b00-4b44-a1bd-1a6c3603398f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:07:54.140757 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700af215-4b00-4b44-a1bd-1a6c3603398f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:07:54.140757 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700af215-4b00-4b44-a1bd-1a6c3603398f None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:07:54.151437 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 627/1279] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 31 01:07:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:07:54.157398 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0231533c-5231-4528-a93a-aa184f3cd233 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:07:54.187954 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:07:54.254610 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0231533c-5231-4528-a93a-aa184f3cd233 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:07:54.286999 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4455daf4-1789-412e-b5d5-bfa13376720c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:07:54.287617 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4455daf4-1789-412e-b5d5-bfa13376720c None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:07:54.287898 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4455daf4-1789-412e-b5d5-bfa13376720c None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:07:54.288174 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4455daf4-1789-412e-b5d5-bfa13376720c None admin] PATH_INFO: `/v3/users/placement` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:07:54.296124 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4455daf4-1789-412e-b5d5-bfa13376720c None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:07:54.297636 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4455daf4-1789-412e-b5d5-bfa13376720c None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:07:54.309102 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application [None req-4455daf4-1789-412e-b5d5-bfa13376720c None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 31 01:07:54.309102 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:07:54.309102 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:07:54.309102 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:07:54.309102 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:07:54.309102 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:07:54.309102 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:07:54.309102 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:07:54.309102 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:07:54.309102 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:07:54.309102 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:07:54.309102 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:07:54.309102 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 01:07:54.309102 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 01:07:54.309102 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 01:07:54.309102 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 01:07:54.313666 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:07:54.313666 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:07:54.313666 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 01:07:54.313666 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 01:07:54.313666 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 01:07:54.313666 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 01:07:54.313666 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:07:54.313666 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:07:54.313666 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:07:54.313666 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:07:54.313666 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:07:54.313666 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application with Lock( Aug 31 01:07:54.313666 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:07:54.313666 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:07:54.313666 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:07:54.313666 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:07:54.313666 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:07:54.313666 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:07:54.314818 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:07:54.314818 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:07:54.314818 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 01:07:54.314818 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 01:07:54.314818 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 01:07:54.314818 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 01:07:54.314818 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 01:07:54.314818 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 01:07:54.314818 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 31 01:07:54.314818 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Aug 31 01:07:54.314818 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 653/1280] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 01:07:54 2022] GET /identity/v3/users/placement => generated 87 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:07:54.316218 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700af215-4b00-4b44-a1bd-1a6c3603398f None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:07:54.320074 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:07:54.414044 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700af215-4b00-4b44-a1bd-1a6c3603398f None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:07:54.447987 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f878eb-2aed-471e-b9ea-c62e2c772ec9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:07:54.448597 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f878eb-2aed-471e-b9ea-c62e2c772ec9 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:07:54.448920 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f878eb-2aed-471e-b9ea-c62e2c772ec9 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:07:54.449200 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f878eb-2aed-471e-b9ea-c62e2c772ec9 None admin] PATH_INFO: `/v3/users` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:07:54.450203 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f878eb-2aed-471e-b9ea-c62e2c772ec9 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:07:54.450511 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f878eb-2aed-471e-b9ea-c62e2c772ec9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:07:54.451956 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f878eb-2aed-471e-b9ea-c62e2c772ec9 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:07:54.469400 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 628/1281] 10.222.0.20 () {56 vars in 1143 bytes} [Wed Aug 31 01:07:54 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:07:54.475315 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4455daf4-1789-412e-b5d5-bfa13376720c None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:07:54.566512 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:07:54.576417 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4455daf4-1789-412e-b5d5-bfa13376720c None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:07:54.608793 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c22552-e8d6-48fd-9433-a4cd33a9cb4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:07:54.609400 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c22552-e8d6-48fd-9433-a4cd33a9cb4e None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:07:54.609684 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c22552-e8d6-48fd-9433-a4cd33a9cb4e None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:07:54.609967 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c22552-e8d6-48fd-9433-a4cd33a9cb4e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:07:54.674638 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c22552-e8d6-48fd-9433-a4cd33a9cb4e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:07:54.676084 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c22552-e8d6-48fd-9433-a4cd33a9cb4e None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:07:54.762197 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:07:54.772117 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 654/1282] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Aug 31 01:07:54 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 01:07:54.778168 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f878eb-2aed-471e-b9ea-c62e2c772ec9 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:07:54.839562 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:07:54.876224 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f878eb-2aed-471e-b9ea-c62e2c772ec9 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:07:54.909693 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4e9709-dd5c-412d-af42-5c0d755e1d88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:07:54.912772 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4e9709-dd5c-412d-af42-5c0d755e1d88 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:07:54.912772 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4e9709-dd5c-412d-af42-5c0d755e1d88 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:07:54.912772 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4e9709-dd5c-412d-af42-5c0d755e1d88 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:07:54.921199 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4e9709-dd5c-412d-af42-5c0d755e1d88 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:07:54.922638 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4e9709-dd5c-412d-af42-5c0d755e1d88 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:07:54.936905 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application [None req-ea4e9709-dd5c-412d-af42-5c0d755e1d88 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 01:07:54.936905 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:07:54.936905 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:07:54.936905 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:07:54.936905 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:07:54.936905 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:07:54.936905 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:07:54.936905 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:07:54.936905 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:07:54.936905 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:07:54.936905 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:07:54.936905 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:07:54.936905 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 01:07:54.936905 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 01:07:54.936905 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 01:07:54.936905 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 01:07:54.938097 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:07:54.938097 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:07:54.938097 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:07:54.938097 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:07:54.938097 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:07:54.938097 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:07:54.938097 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:07:54.938097 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application with Lock( Aug 31 01:07:54.938097 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:07:54.938097 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:07:54.938097 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:07:54.938097 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:07:54.938097 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:07:54.938097 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:07:54.938097 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:07:54.938097 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:07:54.938097 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 01:07:54.939343 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 01:07:54.939343 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 01:07:54.939343 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Aug 31 01:07:54.939343 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 629/1283] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 01:07:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:07:54.943838 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c22552-e8d6-48fd-9433-a4cd33a9cb4e None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:07:55.045410 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c22552-e8d6-48fd-9433-a4cd33a9cb4e None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:07:55.078448 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdef0d84-2dca-446e-b0bd-35b29bea7167 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:07:55.079055 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdef0d84-2dca-446e-b0bd-35b29bea7167 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:07:55.079336 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdef0d84-2dca-446e-b0bd-35b29bea7167 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:07:55.079610 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdef0d84-2dca-446e-b0bd-35b29bea7167 None admin] PATH_INFO: `/v3/domains` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:07:55.080282 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdef0d84-2dca-446e-b0bd-35b29bea7167 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:07:55.080588 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdef0d84-2dca-446e-b0bd-35b29bea7167 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:07:55.082050 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdef0d84-2dca-446e-b0bd-35b29bea7167 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:07:55.093574 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 655/1284] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 31 01:07:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:07:55.099297 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4e9709-dd5c-412d-af42-5c0d755e1d88 None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:07:55.199153 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4e9709-dd5c-412d-af42-5c0d755e1d88 None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:07:55.213473 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:07:55.233631 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee9a86b-a256-4cb9-8649-96b3602bcb2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:07:55.234327 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee9a86b-a256-4cb9-8649-96b3602bcb2c None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:07:55.234611 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee9a86b-a256-4cb9-8649-96b3602bcb2c None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:07:55.234890 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee9a86b-a256-4cb9-8649-96b3602bcb2c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:07:55.241303 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee9a86b-a256-4cb9-8649-96b3602bcb2c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:07:55.242801 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee9a86b-a256-4cb9-8649-96b3602bcb2c None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:07:55.254028 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application [None req-eee9a86b-a256-4cb9-8649-96b3602bcb2c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 01:07:55.254028 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:07:55.254028 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:07:55.254028 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:07:55.254028 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:07:55.254028 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:07:55.254028 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:07:55.254028 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:07:55.254028 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:07:55.254028 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:07:55.254028 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:07:55.254028 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:07:55.254028 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 01:07:55.254028 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 01:07:55.254028 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 01:07:55.255038 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 01:07:55.255038 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:07:55.255038 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:07:55.255038 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:07:55.255038 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:07:55.255038 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:07:55.255038 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:07:55.255038 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:07:55.255038 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application with Lock( Aug 31 01:07:55.255038 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:07:55.255038 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:07:55.255038 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:07:55.255038 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:07:55.255038 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:07:55.255038 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:07:55.255038 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:07:55.255038 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:07:55.255941 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 01:07:55.255941 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 01:07:55.255941 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 01:07:55.255941 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 01:07:55.255941 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 01:07:55.255941 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 01:07:55.255941 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 01:07:55.255941 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: ERROR keystone.server.flask.application Aug 31 01:07:55.255941 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 630/1285] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Aug 31 01:07:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:07:55.261605 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdef0d84-2dca-446e-b0bd-35b29bea7167 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:07:55.312323 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:07:55.364749 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdef0d84-2dca-446e-b0bd-35b29bea7167 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:07:55.404366 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e79390-0687-445a-a198-7fd3e2e00842 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:07:55.405025 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e79390-0687-445a-a198-7fd3e2e00842 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:07:55.405314 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e79390-0687-445a-a198-7fd3e2e00842 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:07:55.405717 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e79390-0687-445a-a198-7fd3e2e00842 None admin] PATH_INFO: `/v3/projects` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:07:55.406427 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e79390-0687-445a-a198-7fd3e2e00842 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:07:55.406741 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e79390-0687-445a-a198-7fd3e2e00842 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:07:55.408191 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e79390-0687-445a-a198-7fd3e2e00842 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:07:55.427133 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 656/1286] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Aug 31 01:07:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:07:55.428604 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee9a86b-a256-4cb9-8649-96b3602bcb2c None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:07:55.541615 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee9a86b-a256-4cb9-8649-96b3602bcb2c None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:07:55.577511 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afe98b2-ae70-41d2-982d-3b58f3ed31fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:07:55.578222 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afe98b2-ae70-41d2-982d-3b58f3ed31fd None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:07:55.578599 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afe98b2-ae70-41d2-982d-3b58f3ed31fd None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:07:55.578923 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afe98b2-ae70-41d2-982d-3b58f3ed31fd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:07:55.579713 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afe98b2-ae70-41d2-982d-3b58f3ed31fd None admin] RBAC: Adding query filter params (role.id=d009411496b8452b88e77ffb8db015a7, scope.project.id=60b2abc15c2849ba8a1ed92117cc981b, user.id=dea0fe5b1d6b405cb47b95192580303b) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:07:55.580070 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afe98b2-ae70-41d2-982d-3b58f3ed31fd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:07:55.581671 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afe98b2-ae70-41d2-982d-3b58f3ed31fd None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:07:55.589183 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 631/1287] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Aug 31 01:07:55 2022] GET /identity/v3/role_assignments?user.id=dea0fe5b1d6b405cb47b95192580303b&scope.project.id=60b2abc15c2849ba8a1ed92117cc981b&role.id=d009411496b8452b88e77ffb8db015a7 => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:07:57.538524 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b321e8a9-cfc8-43e0-92cd-1a7109b4d053 None None] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:07:57.539056 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b321e8a9-cfc8-43e0-92cd-1a7109b4d053 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:07:57.539056 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b321e8a9-cfc8-43e0-92cd-1a7109b4d053 None None] PATH_INFO: `/` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:07:57.540945 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 657/1288] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:07:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:07:57.546129 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d72b6e5-da44-45aa-b78b-454418c2b6eb None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:07:57.546532 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d72b6e5-da44-45aa-b78b-454418c2b6eb None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:07:57.546684 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d72b6e5-da44-45aa-b78b-454418c2b6eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:07:57.565308 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-5d72b6e5-da44-45aa-b78b-454418c2b6eb None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:07:57.566490 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-5d72b6e5-da44-45aa-b78b-454418c2b6eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:07:57.595076 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:07:57.631132 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 632/1289] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:07:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 01:07:57.893577 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b321e8a9-cfc8-43e0-92cd-1a7109b4d053 None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:07:57.994804 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b321e8a9-cfc8-43e0-92cd-1a7109b4d053 None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:07:58.005920 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:07:58.029120 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc54eca-828a-424e-951e-103602fda354 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:07:58.029744 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc54eca-828a-424e-951e-103602fda354 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:07:58.030026 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc54eca-828a-424e-951e-103602fda354 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:07:58.030299 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc54eca-828a-424e-951e-103602fda354 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:07:58.030875 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc54eca-828a-424e-951e-103602fda354 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:07:58.032237 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc54eca-828a-424e-951e-103602fda354 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:07:58.041754 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application [None req-3fc54eca-828a-424e-951e-103602fda354 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 31 01:07:58.041754 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:07:58.041754 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 01:07:58.041754 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 01:07:58.041754 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 01:07:58.041754 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 01:07:58.041754 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 01:07:58.041754 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 01:07:58.041754 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 31 01:07:58.041754 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Aug 31 01:07:58.041754 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 01:07:58.041754 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Aug 31 01:07:58.041754 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:07:58.041754 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:07:58.041754 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:07:58.041754 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:07:58.041754 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:07:58.042847 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:07:58.042847 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:07:58.042847 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:07:58.042847 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:07:58.042847 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:07:58.042847 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:07:58.042847 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 01:07:58.042847 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 01:07:58.042847 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 01:07:58.042847 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 01:07:58.042847 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:07:58.042847 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:07:58.042847 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:07:58.042847 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:07:58.042847 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:07:58.042847 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:07:58.043726 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:07:58.043726 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application with Lock( Aug 31 01:07:58.043726 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:07:58.043726 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:07:58.043726 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:07:58.043726 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:07:58.043726 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:07:58.043726 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:07:58.043726 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:07:58.043726 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:07:58.043726 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 01:07:58.043726 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 01:07:58.043726 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 31 01:07:58.043726 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Aug 31 01:07:58.043726 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 658/1290] 10.222.0.20 () {56 vars in 1117 bytes} [Wed Aug 31 01:07:57 2022] GET /identity/v3/services/placement => generated 90 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:07:58.049048 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d72b6e5-da44-45aa-b78b-454418c2b6eb None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:07:58.147494 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d72b6e5-da44-45aa-b78b-454418c2b6eb None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:07:58.181776 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f751bc9-db81-4c53-aef9-e42c256d7860 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:07:58.182383 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f751bc9-db81-4c53-aef9-e42c256d7860 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:07:58.182687 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f751bc9-db81-4c53-aef9-e42c256d7860 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:07:58.182962 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f751bc9-db81-4c53-aef9-e42c256d7860 None admin] PATH_INFO: `/v3/services` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:07:58.183590 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f751bc9-db81-4c53-aef9-e42c256d7860 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:07:58.183893 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f751bc9-db81-4c53-aef9-e42c256d7860 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:07:58.185301 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f751bc9-db81-4c53-aef9-e42c256d7860 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:07:58.189981 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 633/1291] 10.222.0.20 () {56 vars in 1116 bytes} [Wed Aug 31 01:07:58 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:07:58.195672 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc54eca-828a-424e-951e-103602fda354 None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:07:58.299743 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc54eca-828a-424e-951e-103602fda354 None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:07:58.332447 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbfd8ca-cf9b-4e61-b5fe-2b104fdd6fd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:07:58.333080 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbfd8ca-cf9b-4e61-b5fe-2b104fdd6fd9 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:07:58.333363 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbfd8ca-cf9b-4e61-b5fe-2b104fdd6fd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:07:58.333636 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbfd8ca-cf9b-4e61-b5fe-2b104fdd6fd9 None admin] PATH_INFO: `/v3/services` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:07:58.334316 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbfd8ca-cf9b-4e61-b5fe-2b104fdd6fd9 None admin] RBAC: Adding query filter params (type=placement) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:07:58.334621 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbfd8ca-cf9b-4e61-b5fe-2b104fdd6fd9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:07:58.335977 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbfd8ca-cf9b-4e61-b5fe-2b104fdd6fd9 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:07:58.341050 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 659/1292] 10.222.0.20 () {56 vars in 1116 bytes} [Wed Aug 31 01:07:58 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:08:00.209792 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb032b0-66fe-4e3f-a4e4-baa740a21b58 None None] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:08:00.210315 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb032b0-66fe-4e3f-a4e4-baa740a21b58 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:08:00.210407 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb032b0-66fe-4e3f-a4e4-baa740a21b58 None None] PATH_INFO: `/` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:08:00.211744 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 634/1293] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:08:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:08:00.217025 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560b2714-a145-4f1e-9e0e-9dc48126c576 None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:08:00.217443 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560b2714-a145-4f1e-9e0e-9dc48126c576 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:08:00.217577 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560b2714-a145-4f1e-9e0e-9dc48126c576 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:08:00.235309 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-560b2714-a145-4f1e-9e0e-9dc48126c576 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:08:00.236490 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-560b2714-a145-4f1e-9e0e-9dc48126c576 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:08:00.265976 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:08:00.301942 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 660/1294] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:08:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 01:08:00.575330 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb032b0-66fe-4e3f-a4e4-baa740a21b58 None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:08:00.606484 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:08:00.673361 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb032b0-66fe-4e3f-a4e4-baa740a21b58 None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:08:00.708088 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa95454b-58ab-43d7-93cc-aaf3a55c2863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:08:00.708673 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa95454b-58ab-43d7-93cc-aaf3a55c2863 None admin] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:08:00.708988 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa95454b-58ab-43d7-93cc-aaf3a55c2863 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:08:00.709266 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa95454b-58ab-43d7-93cc-aaf3a55c2863 None admin] PATH_INFO: `/v3/services` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:08:00.709906 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa95454b-58ab-43d7-93cc-aaf3a55c2863 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:08:00.711271 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa95454b-58ab-43d7-93cc-aaf3a55c2863 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:08:00.801018 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 635/1295] 10.222.0.20 () {60 vars in 1141 bytes} [Wed Aug 31 01:08:00 2022] POST /identity/v3/services => generated 246 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:08:02.694206 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da79548-0e50-486d-acc9-9a94eedd928b None None] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:08:02.694768 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da79548-0e50-486d-acc9-9a94eedd928b None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:08:02.694768 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da79548-0e50-486d-acc9-9a94eedd928b None None] PATH_INFO: `/` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:08:02.696078 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 661/1296] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:08:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:08:02.701806 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4de653-f9f6-46d2-8819-455424f35ab8 None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:08:02.702228 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4de653-f9f6-46d2-8819-455424f35ab8 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:08:02.702348 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4de653-f9f6-46d2-8819-455424f35ab8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:08:02.720883 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-8a4de653-f9f6-46d2-8819-455424f35ab8 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:08:02.722092 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-8a4de653-f9f6-46d2-8819-455424f35ab8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:08:02.792637 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 636/1297] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:08:02 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 01:08:03.055933 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da79548-0e50-486d-acc9-9a94eedd928b None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:08:03.155604 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da79548-0e50-486d-acc9-9a94eedd928b None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:08:03.187867 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9f3c7a-0788-41a6-80a1-0043438ed0ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:08:03.188479 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9f3c7a-0788-41a6-80a1-0043438ed0ff None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:08:03.188778 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9f3c7a-0788-41a6-80a1-0043438ed0ff None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:08:03.189050 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9f3c7a-0788-41a6-80a1-0043438ed0ff None admin] PATH_INFO: `/v3/services/placement` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:08:03.189617 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9f3c7a-0788-41a6-80a1-0043438ed0ff None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:08:03.190966 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9f3c7a-0788-41a6-80a1-0043438ed0ff None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:08:03.200265 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application [None req-9d9f3c7a-0788-41a6-80a1-0043438ed0ff None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 31 01:08:03.200265 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:08:03.200265 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 01:08:03.200265 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 01:08:03.200265 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 01:08:03.200265 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 01:08:03.200265 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 01:08:03.200265 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 01:08:03.200265 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 31 01:08:03.200265 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Aug 31 01:08:03.200265 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 01:08:03.200265 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Aug 31 01:08:03.200265 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:08:03.200265 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:08:03.200265 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:08:03.200265 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:08:03.200265 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:08:03.201497 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:08:03.201497 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:08:03.201497 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:08:03.201497 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:08:03.201497 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:08:03.201497 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:08:03.201497 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 01:08:03.201497 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 01:08:03.201497 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 01:08:03.201497 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 01:08:03.201497 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:08:03.201497 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:08:03.201497 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:08:03.201497 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:08:03.201497 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:08:03.201497 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:08:03.202512 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:08:03.202512 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application with Lock( Aug 31 01:08:03.202512 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:08:03.202512 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:08:03.202512 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:08:03.202512 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:08:03.202512 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:08:03.202512 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:08:03.202512 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:08:03.202512 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:08:03.202512 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 01:08:03.202512 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 01:08:03.202512 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 31 01:08:03.202512 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Aug 31 01:08:03.202512 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 662/1298] 10.222.0.20 () {56 vars in 1117 bytes} [Wed Aug 31 01:08:03 2022] GET /identity/v3/services/placement => generated 90 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:08:03.207359 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4de653-f9f6-46d2-8819-455424f35ab8 None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:08:03.225195 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:08:03.308520 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4de653-f9f6-46d2-8819-455424f35ab8 None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:08:03.343371 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef5b26e-377b-4ec1-ad30-66acf013e19e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:08:03.343979 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef5b26e-377b-4ec1-ad30-66acf013e19e None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:08:03.344264 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef5b26e-377b-4ec1-ad30-66acf013e19e None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:08:03.344539 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef5b26e-377b-4ec1-ad30-66acf013e19e None admin] PATH_INFO: `/v3/services` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:08:03.345199 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef5b26e-377b-4ec1-ad30-66acf013e19e None admin] RBAC: Adding query filter params (name=placement) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:08:03.345505 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef5b26e-377b-4ec1-ad30-66acf013e19e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:08:03.346877 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef5b26e-377b-4ec1-ad30-66acf013e19e None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:08:03.351962 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 637/1299] 10.222.0.20 () {56 vars in 1116 bytes} [Wed Aug 31 01:08:03 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:08:03.358009 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9f3c7a-0788-41a6-80a1-0043438ed0ff None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:08:03.389528 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:08:03.458398 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9f3c7a-0788-41a6-80a1-0043438ed0ff None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:08:03.491224 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fc2690-b1ff-4a1c-840c-b7e5342182c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:08:03.491825 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fc2690-b1ff-4a1c-840c-b7e5342182c9 None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:08:03.492104 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fc2690-b1ff-4a1c-840c-b7e5342182c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:08:03.492380 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fc2690-b1ff-4a1c-840c-b7e5342182c9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:08:03.493731 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fc2690-b1ff-4a1c-840c-b7e5342182c9 None admin] RBAC: Adding query filter params (interface=public, service_id=68baca9bf8294e4dbdde2acfb806a5ec, region_id=RegionOne) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:08:03.494039 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fc2690-b1ff-4a1c-840c-b7e5342182c9 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:08:03.495391 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fc2690-b1ff-4a1c-840c-b7e5342182c9 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:08:03.500286 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 663/1300] 10.222.0.20 () {56 vars in 1251 bytes} [Wed Aug 31 01:08:03 2022] GET /identity/v3/endpoints?service_id=68baca9bf8294e4dbdde2acfb806a5ec&interface=public®ion_id=RegionOne => generated 194 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:08:03.506213 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef5b26e-377b-4ec1-ad30-66acf013e19e None admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:08:03.586708 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:08:03.604996 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef5b26e-377b-4ec1-ad30-66acf013e19e None admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:08:03.638834 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b30d760-59a7-492e-bd61-1ae951ec4db5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:08:03.639425 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b30d760-59a7-492e-bd61-1ae951ec4db5 None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:08:03.639704 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b30d760-59a7-492e-bd61-1ae951ec4db5 None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:08:03.639972 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b30d760-59a7-492e-bd61-1ae951ec4db5 None admin] PATH_INFO: `/v3/services` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:08:03.640559 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b30d760-59a7-492e-bd61-1ae951ec4db5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:08:03.641957 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b30d760-59a7-492e-bd61-1ae951ec4db5 None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:08:03.648522 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 638/1301] 10.222.0.20 () {56 vars in 1087 bytes} [Wed Aug 31 01:08:03 2022] GET /identity/v3/services => generated 2191 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:08:05.534801 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcbf4ce-ce16-4af8-ae26-5f97c2f7b86e None None] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:08:05.535361 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcbf4ce-ce16-4af8-ae26-5f97c2f7b86e None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:08:05.535361 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcbf4ce-ce16-4af8-ae26-5f97c2f7b86e None None] PATH_INFO: `/` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:08:05.536690 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 664/1302] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:08:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:08:05.549156 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988bed39-0fbc-47b5-9ba8-14c71c8a154b None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:08:05.549601 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988bed39-0fbc-47b5-9ba8-14c71c8a154b None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:08:05.549704 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988bed39-0fbc-47b5-9ba8-14c71c8a154b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:08:05.568089 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-988bed39-0fbc-47b5-9ba8-14c71c8a154b None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:08:05.569318 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-988bed39-0fbc-47b5-9ba8-14c71c8a154b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:08:05.635572 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 639/1303] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:08:05 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 01:08:05.909145 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcbf4ce-ce16-4af8-ae26-5f97c2f7b86e None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:08:06.002840 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:08:06.010220 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcbf4ce-ce16-4af8-ae26-5f97c2f7b86e None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:08:06.044327 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdccf14f-0ede-443a-805a-8b315d60d35b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:08:06.045118 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdccf14f-0ede-443a-805a-8b315d60d35b None admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:08:06.045407 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdccf14f-0ede-443a-805a-8b315d60d35b None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:08:06.045685 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdccf14f-0ede-443a-805a-8b315d60d35b None admin] PATH_INFO: `/v3/services/placement` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:08:06.046263 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdccf14f-0ede-443a-805a-8b315d60d35b None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:08:06.047624 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdccf14f-0ede-443a-805a-8b315d60d35b None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:08:06.056428 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application [None req-bdccf14f-0ede-443a-805a-8b315d60d35b None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 31 01:08:06.056428 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:08:06.056428 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 01:08:06.056428 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 01:08:06.056428 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 01:08:06.056428 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 01:08:06.056428 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 01:08:06.056428 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 01:08:06.056428 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 31 01:08:06.056428 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Aug 31 01:08:06.056428 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 01:08:06.056428 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Aug 31 01:08:06.056428 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 01:08:06.056428 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 01:08:06.056428 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 01:08:06.056428 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 01:08:06.056428 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 01:08:06.057747 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 01:08:06.057747 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 01:08:06.057747 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 01:08:06.057747 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 01:08:06.057747 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 01:08:06.057747 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 01:08:06.057747 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 01:08:06.057747 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 01:08:06.057747 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 01:08:06.057747 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 01:08:06.057747 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 01:08:06.057747 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 01:08:06.057747 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 01:08:06.057747 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 01:08:06.057747 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 01:08:06.057747 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 01:08:06.058882 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 01:08:06.058882 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application with Lock( Aug 31 01:08:06.058882 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 01:08:06.058882 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self._enter() Aug 31 01:08:06.058882 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 01:08:06.058882 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 01:08:06.058882 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 01:08:06.058882 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application return self.creator() Aug 31 01:08:06.058882 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 01:08:06.058882 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application created_value = creator( Aug 31 01:08:06.058882 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 01:08:06.058882 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 01:08:06.058882 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 31 01:08:06.058882 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: ERROR keystone.server.flask.application Aug 31 01:08:06.058882 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 665/1304] 10.222.0.20 () {56 vars in 1117 bytes} [Wed Aug 31 01:08:05 2022] GET /identity/v3/services/placement => generated 90 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 01:08:06.063506 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988bed39-0fbc-47b5-9ba8-14c71c8a154b None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:08:06.139595 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:08:06.161019 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988bed39-0fbc-47b5-9ba8-14c71c8a154b None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:08:06.193850 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea87f63-5822-46b5-af10-9ee1839a22ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:08:06.194486 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea87f63-5822-46b5-af10-9ee1839a22ed None admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:08:06.194764 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea87f63-5822-46b5-af10-9ee1839a22ed None admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:08:06.195034 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea87f63-5822-46b5-af10-9ee1839a22ed None admin] PATH_INFO: `/v3/services` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:08:06.195653 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea87f63-5822-46b5-af10-9ee1839a22ed None admin] RBAC: Adding query filter params (name=placement) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:08:06.195953 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea87f63-5822-46b5-af10-9ee1839a22ed None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:08:06.197955 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea87f63-5822-46b5-af10-9ee1839a22ed None admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:08:06.204084 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 640/1305] 10.222.0.20 () {56 vars in 1116 bytes} [Wed Aug 31 01:08:06 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:08:06.209911 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdccf14f-0ede-443a-805a-8b315d60d35b None admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:08:06.308681 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdccf14f-0ede-443a-805a-8b315d60d35b None admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:08:06.341772 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6175eb-17c4-48fa-9631-645a161416a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:08:06.342402 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6175eb-17c4-48fa-9631-645a161416a0 None admin] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:08:06.342718 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6175eb-17c4-48fa-9631-645a161416a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:08:06.343017 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6175eb-17c4-48fa-9631-645a161416a0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:08:06.343693 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6175eb-17c4-48fa-9631-645a161416a0 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:08:06.345092 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6175eb-17c4-48fa-9631-645a161416a0 None admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:08:06.435665 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 666/1306] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 01:08:06 2022] POST /identity/v3/endpoints => generated 331 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:08:16.201541 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=30 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:08:16.430471 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:14:04.453952 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c69cb2-44a7-4d05-ae1a-38f3fa2eb82f None None] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:14:04.454637 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c69cb2-44a7-4d05-ae1a-38f3fa2eb82f None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:14:04.454841 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c69cb2-44a7-4d05-ae1a-38f3fa2eb82f None None] PATH_INFO: `/` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:14:04.456288 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 641/1307] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:14:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:14:04.470552 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf12044-b777-4c42-9b45-824b4f96123a None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:14:04.470552 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf12044-b777-4c42-9b45-824b4f96123a None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:14:04.470552 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf12044-b777-4c42-9b45-824b4f96123a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:14:04.552658 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-1bf12044-b777-4c42-9b45-824b4f96123a None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:14:04.557480 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-1bf12044-b777-4c42-9b45-824b4f96123a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:14:04.685629 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 667/1308] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:14:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:14:04.974431 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba5e86b-049a-41cc-a75c-03d245bd012d None None] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:14:04.974720 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba5e86b-049a-41cc-a75c-03d245bd012d None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:14:04.974996 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba5e86b-049a-41cc-a75c-03d245bd012d None None] PATH_INFO: `/` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:14:04.976375 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 642/1309] 10.222.0.20 () {54 vars in 965 bytes} [Wed Aug 31 01:14:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:14:04.982537 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fc2600-9347-4d7e-962a-c39c8c0bee14 None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:14:04.982833 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fc2600-9347-4d7e-962a-c39c8c0bee14 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:14:04.983163 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fc2600-9347-4d7e-962a-c39c8c0bee14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:14:05.060709 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-a4fc2600-9347-4d7e-962a-c39c8c0bee14 None None] MFA Rules not processed for user `0817c3320d5d41d8bba0550c8ae8dd60`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:14:05.061969 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-a4fc2600-9347-4d7e-962a-c39c8c0bee14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:14:05.084641 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:14:05.151231 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 668/1310] 10.222.0.20 () {58 vars in 1062 bytes} [Wed Aug 31 01:14:04 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:14:05.160352 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba5e86b-049a-41cc-a75c-03d245bd012d None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:14:05.162214 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-1ba5e86b-049a-41cc-a75c-03d245bd012d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:14:05.282883 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba5e86b-049a-41cc-a75c-03d245bd012d None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:14:05.360338 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b45cf7d-94cb-4097-aff7-b7d8a0d9406e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0817c3320d5d41d8bba0550c8ae8dd60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:14:05.360977 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b45cf7d-94cb-4097-aff7-b7d8a0d9406e service neutron] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:14:05.361257 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b45cf7d-94cb-4097-aff7-b7d8a0d9406e service neutron] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:14:05.361525 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b45cf7d-94cb-4097-aff7-b7d8a0d9406e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:14:05.363129 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-5b45cf7d-94cb-4097-aff7-b7d8a0d9406e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:14:05.420608 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:14:05.476652 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b45cf7d-94cb-4097-aff7-b7d8a0d9406e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:14:05.478143 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b45cf7d-94cb-4097-aff7-b7d8a0d9406e service neutron] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:14:05.497514 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 643/1311] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:14:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:14:07.700452 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73ce2ee-1166-4244-b472-591da72b175f None None] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:14:07.701043 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73ce2ee-1166-4244-b472-591da72b175f None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:14:07.701043 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73ce2ee-1166-4244-b472-591da72b175f None None] PATH_INFO: `/` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:14:07.702543 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 669/1312] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:14:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 01:14:07.708135 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d9af34-5a88-4b31-86ab-f0e22c8fa6e5 None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:14:07.708620 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d9af34-5a88-4b31-86ab-f0e22c8fa6e5 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:14:07.708702 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d9af34-5a88-4b31-86ab-f0e22c8fa6e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:14:07.789999 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-72d9af34-5a88-4b31-86ab-f0e22c8fa6e5 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:14:07.791213 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-72d9af34-5a88-4b31-86ab-f0e22c8fa6e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:14:07.904211 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 644/1313] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:14:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:14:08.183860 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73ce2ee-1166-4244-b472-591da72b175f None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:14:08.185482 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-b73ce2ee-1166-4244-b472-591da72b175f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:14:08.300841 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73ce2ee-1166-4244-b472-591da72b175f None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:14:08.375551 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:14:08.382160 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5bb6e1-67b0-4818-8d10-ae850e0b5fc3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0817c3320d5d41d8bba0550c8ae8dd60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:14:08.382873 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5bb6e1-67b0-4818-8d10-ae850e0b5fc3 service neutron] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:14:08.383222 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5bb6e1-67b0-4818-8d10-ae850e0b5fc3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:14:08.383580 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5bb6e1-67b0-4818-8d10-ae850e0b5fc3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:14:08.386353 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-0b5bb6e1-67b0-4818-8d10-ae850e0b5fc3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:14:08.507119 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5bb6e1-67b0-4818-8d10-ae850e0b5fc3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:14:08.508490 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5bb6e1-67b0-4818-8d10-ae850e0b5fc3 service neutron] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:14:08.528143 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 670/1314] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:14:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:14:10.940874 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebb6259-d324-44a4-a921-270e29525b0b None None] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:14:10.941342 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebb6259-d324-44a4-a921-270e29525b0b None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:14:10.941437 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebb6259-d324-44a4-a921-270e29525b0b None None] PATH_INFO: `/` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:14:10.942778 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 645/1315] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:14:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:14:10.948308 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e710ed-1ef0-4f18-827c-9dd3dc76fd3c None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:14:10.948777 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e710ed-1ef0-4f18-827c-9dd3dc76fd3c None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:14:10.948896 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e710ed-1ef0-4f18-827c-9dd3dc76fd3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:14:11.015358 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-d6e710ed-1ef0-4f18-827c-9dd3dc76fd3c None None] MFA Rules not processed for user `1aaea1e3d940441eb7f58357392a5940`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:14:11.016576 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-d6e710ed-1ef0-4f18-827c-9dd3dc76fd3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:14:11.066305 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:14:11.110046 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 671/1316] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:14:10 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:14:11.392550 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebb6259-d324-44a4-a921-270e29525b0b None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:14:11.394133 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-0ebb6259-d324-44a4-a921-270e29525b0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:14:11.403869 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:14:11.497212 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebb6259-d324-44a4-a921-270e29525b0b None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:14:11.570227 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83f061e-3404-40eb-8960-0a497d713ee2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0817c3320d5d41d8bba0550c8ae8dd60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:14:11.570842 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83f061e-3404-40eb-8960-0a497d713ee2 service neutron] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:14:11.571124 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83f061e-3404-40eb-8960-0a497d713ee2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:14:11.571398 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83f061e-3404-40eb-8960-0a497d713ee2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:14:11.573019 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-c83f061e-3404-40eb-8960-0a497d713ee2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:14:11.634875 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:14:11.681234 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83f061e-3404-40eb-8960-0a497d713ee2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:14:11.682666 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83f061e-3404-40eb-8960-0a497d713ee2 service neutron] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:14:11.700278 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 646/1317] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:14:11 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:14:14.415792 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a5b695-6e54-4ba6-bf85-f94b621836ad None None] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:14:14.416363 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a5b695-6e54-4ba6-bf85-f94b621836ad None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:14:14.416363 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a5b695-6e54-4ba6-bf85-f94b621836ad None None] PATH_INFO: `/` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:14:14.417735 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 672/1318] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:14:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:14:14.422811 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4076db-8205-4e1c-ac02-28c7cd3c36b0 None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:14:14.423208 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4076db-8205-4e1c-ac02-28c7cd3c36b0 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:14:14.423373 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4076db-8205-4e1c-ac02-28c7cd3c36b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:14:14.493212 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-1d4076db-8205-4e1c-ac02-28c7cd3c36b0 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:14:14.494737 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-1d4076db-8205-4e1c-ac02-28c7cd3c36b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:14:14.598576 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 647/1319] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:14:14 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:14:14.894711 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a5b695-6e54-4ba6-bf85-f94b621836ad None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:14:14.897280 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-80a5b695-6e54-4ba6-bf85-f94b621836ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:14:14.997173 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a5b695-6e54-4ba6-bf85-f94b621836ad None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:14:15.068800 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a6897c-9036-4620-b045-0ffccf53c681 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0817c3320d5d41d8bba0550c8ae8dd60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:14:15.069406 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a6897c-9036-4620-b045-0ffccf53c681 service neutron] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:14:15.069683 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a6897c-9036-4620-b045-0ffccf53c681 service neutron] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:14:15.069954 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a6897c-9036-4620-b045-0ffccf53c681 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:14:15.071531 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-76a6897c-9036-4620-b045-0ffccf53c681 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:14:15.078805 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:14:15.175171 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a6897c-9036-4620-b045-0ffccf53c681 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:14:15.176600 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a6897c-9036-4620-b045-0ffccf53c681 service neutron] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:14:15.195577 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 673/1320] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:14:14 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 01:14:18.057541 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ea42a5-f421-4a33-ad87-835413ebce7c None None] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:14:18.058096 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ea42a5-f421-4a33-ad87-835413ebce7c None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:14:18.058096 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ea42a5-f421-4a33-ad87-835413ebce7c None None] PATH_INFO: `/` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:14:18.059574 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 648/1321] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:14:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:14:18.064510 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f3d079-9713-48bc-98f7-e0d827860b79 None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:14:18.064939 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f3d079-9713-48bc-98f7-e0d827860b79 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:14:18.065093 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f3d079-9713-48bc-98f7-e0d827860b79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:14:18.129627 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-83f3d079-9713-48bc-98f7-e0d827860b79 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:14:18.131199 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-83f3d079-9713-48bc-98f7-e0d827860b79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:14:18.187326 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:14:18.226150 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 674/1322] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:14:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:14:18.507801 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ea42a5-f421-4a33-ad87-835413ebce7c None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:14:18.509382 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-37ea42a5-f421-4a33-ad87-835413ebce7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:14:18.530461 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:14:18.618590 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ea42a5-f421-4a33-ad87-835413ebce7c None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:14:18.694125 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc8e8a7-ce42-4c96-9dd2-b0071f8f0a81 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0817c3320d5d41d8bba0550c8ae8dd60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:14:18.694729 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc8e8a7-ce42-4c96-9dd2-b0071f8f0a81 service neutron] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:14:18.695010 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc8e8a7-ce42-4c96-9dd2-b0071f8f0a81 service neutron] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:14:18.695285 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc8e8a7-ce42-4c96-9dd2-b0071f8f0a81 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:14:18.696982 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-7dc8e8a7-ce42-4c96-9dd2-b0071f8f0a81 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:14:18.768532 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:14:18.805511 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc8e8a7-ce42-4c96-9dd2-b0071f8f0a81 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:14:18.807292 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc8e8a7-ce42-4c96-9dd2-b0071f8f0a81 service neutron] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:14:18.825258 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 649/1323] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:14:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:14:20.790932 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b0b449-c98f-4e66-95b3-cfa7f001b803 None None] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:14:20.791486 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b0b449-c98f-4e66-95b3-cfa7f001b803 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:14:20.791486 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b0b449-c98f-4e66-95b3-cfa7f001b803 None None] PATH_INFO: `/` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:14:20.792844 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 675/1324] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:14:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:14:20.798003 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c6d14d-9fa6-440b-89e5-59ef6ebcb099 None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:14:20.798430 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c6d14d-9fa6-440b-89e5-59ef6ebcb099 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:14:20.798557 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c6d14d-9fa6-440b-89e5-59ef6ebcb099 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:14:20.863531 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-d0c6d14d-9fa6-440b-89e5-59ef6ebcb099 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:14:20.865409 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-d0c6d14d-9fa6-440b-89e5-59ef6ebcb099 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:14:20.963382 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 650/1325] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:14:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:14:21.242328 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b0b449-c98f-4e66-95b3-cfa7f001b803 None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:14:21.243908 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-60b0b449-c98f-4e66-95b3-cfa7f001b803 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:14:21.352392 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b0b449-c98f-4e66-95b3-cfa7f001b803 None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:14:21.424448 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cef1cc-43f5-45bd-af14-505704341408 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0817c3320d5d41d8bba0550c8ae8dd60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:14:21.425093 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cef1cc-43f5-45bd-af14-505704341408 service neutron] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:14:21.425374 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cef1cc-43f5-45bd-af14-505704341408 service neutron] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:14:21.425650 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cef1cc-43f5-45bd-af14-505704341408 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:14:21.427275 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-01cef1cc-43f5-45bd-af14-505704341408 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:14:21.450239 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:14:21.548855 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cef1cc-43f5-45bd-af14-505704341408 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:14:21.550404 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cef1cc-43f5-45bd-af14-505704341408 service neutron] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:14:21.569636 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 676/1326] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:14:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:14:23.512295 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef24505-6770-49b5-b16a-5454b99159da None None] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:14:23.512867 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef24505-6770-49b5-b16a-5454b99159da None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:14:23.512867 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef24505-6770-49b5-b16a-5454b99159da None None] PATH_INFO: `/` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:14:23.514231 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 651/1327] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:14:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:14:23.519384 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e558587-dc5a-4fcd-b67c-f5e535a59b7a None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:14:23.519817 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e558587-dc5a-4fcd-b67c-f5e535a59b7a None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:14:23.519922 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e558587-dc5a-4fcd-b67c-f5e535a59b7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:14:23.592701 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-5e558587-dc5a-4fcd-b67c-f5e535a59b7a None None] MFA Rules not processed for user `1aaea1e3d940441eb7f58357392a5940`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:14:23.593916 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-5e558587-dc5a-4fcd-b67c-f5e535a59b7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:14:23.658977 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:14:23.690826 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 677/1328] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:14:23 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:14:23.987627 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab286ed9-4239-4cae-bd46-763e47b9c3ce None None] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:14:23.987918 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab286ed9-4239-4cae-bd46-763e47b9c3ce None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:14:23.988229 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab286ed9-4239-4cae-bd46-763e47b9c3ce None None] PATH_INFO: `/` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:14:23.989622 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 652/1329] 10.222.0.20 () {54 vars in 965 bytes} [Wed Aug 31 01:14:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:14:23.995488 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef697c6-f846-4182-94f1-580204da836e None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:14:23.995774 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef697c6-f846-4182-94f1-580204da836e None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:14:23.996053 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef697c6-f846-4182-94f1-580204da836e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:14:24.072274 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-1ef697c6-f846-4182-94f1-580204da836e None None] MFA Rules not processed for user `0817c3320d5d41d8bba0550c8ae8dd60`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:14:24.073528 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-1ef697c6-f846-4182-94f1-580204da836e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:14:24.163335 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 678/1330] 10.222.0.20 () {58 vars in 1062 bytes} [Wed Aug 31 01:14:23 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 01:14:24.172609 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab286ed9-4239-4cae-bd46-763e47b9c3ce None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:14:24.174164 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-ab286ed9-4239-4cae-bd46-763e47b9c3ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:14:24.208445 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:14:24.279695 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab286ed9-4239-4cae-bd46-763e47b9c3ce None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:14:24.352367 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea2d769-f191-4df1-bbcc-23b0c39f9951 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0817c3320d5d41d8bba0550c8ae8dd60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:14:24.352932 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea2d769-f191-4df1-bbcc-23b0c39f9951 service neutron] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:14:24.353219 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea2d769-f191-4df1-bbcc-23b0c39f9951 service neutron] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:14:24.353498 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea2d769-f191-4df1-bbcc-23b0c39f9951 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:14:24.355126 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-fea2d769-f191-4df1-bbcc-23b0c39f9951 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:14:24.434488 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:14:24.465997 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea2d769-f191-4df1-bbcc-23b0c39f9951 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:14:24.467494 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea2d769-f191-4df1-bbcc-23b0c39f9951 service neutron] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:14:24.486958 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 653/1331] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:14:24 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:14:27.306541 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6aff832-e062-4211-a851-b7c20ba89f47 None None] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:14:27.307082 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6aff832-e062-4211-a851-b7c20ba89f47 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:14:27.307082 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6aff832-e062-4211-a851-b7c20ba89f47 None None] PATH_INFO: `/` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:14:27.308447 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 679/1332] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:14:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:14:27.313696 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf0a900-c451-4a3a-8831-cf17d3ee8f43 None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:14:27.314092 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf0a900-c451-4a3a-8831-cf17d3ee8f43 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:14:27.314255 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf0a900-c451-4a3a-8831-cf17d3ee8f43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:14:27.380046 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-2cf0a900-c451-4a3a-8831-cf17d3ee8f43 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:14:27.381280 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-2cf0a900-c451-4a3a-8831-cf17d3ee8f43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:14:27.477000 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 654/1333] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:14:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:14:27.758061 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6aff832-e062-4211-a851-b7c20ba89f47 None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:14:27.759606 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-a6aff832-e062-4211-a851-b7c20ba89f47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:14:27.827325 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:14:27.860256 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6aff832-e062-4211-a851-b7c20ba89f47 None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:14:27.932954 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0e1f4b-2e02-49be-b599-816db93042ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0817c3320d5d41d8bba0550c8ae8dd60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:14:27.933584 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0e1f4b-2e02-49be-b599-816db93042ca service neutron] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:14:27.933850 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0e1f4b-2e02-49be-b599-816db93042ca service neutron] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:14:27.934125 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0e1f4b-2e02-49be-b599-816db93042ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:14:27.935706 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-db0e1f4b-2e02-49be-b599-816db93042ca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:14:28.041157 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0e1f4b-2e02-49be-b599-816db93042ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:14:28.042517 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0e1f4b-2e02-49be-b599-816db93042ca service neutron] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:14:28.052814 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:14:28.060629 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 680/1334] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:14:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:14:30.034521 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7b516c-d4e4-433b-bf65-e807568c511c None None] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:14:30.035036 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7b516c-d4e4-433b-bf65-e807568c511c None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:14:30.035154 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7b516c-d4e4-433b-bf65-e807568c511c None None] PATH_INFO: `/` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:14:30.036404 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 655/1335] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:14:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:14:30.041639 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d94fce-ddf2-4d6c-b82d-11637d3f34e0 None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:14:30.042036 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d94fce-ddf2-4d6c-b82d-11637d3f34e0 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:14:30.042201 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d94fce-ddf2-4d6c-b82d-11637d3f34e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:14:30.107583 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-b8d94fce-ddf2-4d6c-b82d-11637d3f34e0 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:14:30.108838 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-b8d94fce-ddf2-4d6c-b82d-11637d3f34e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:14:30.203877 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 681/1336] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:14:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:14:30.488838 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7b516c-d4e4-433b-bf65-e807568c511c None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:14:30.490950 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-7c7b516c-d4e4-433b-bf65-e807568c511c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:14:30.530131 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:14:30.593443 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7b516c-d4e4-433b-bf65-e807568c511c None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:14:30.665678 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96d3c28-490d-4632-b519-6eedcf711ef9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0817c3320d5d41d8bba0550c8ae8dd60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:14:30.666284 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96d3c28-490d-4632-b519-6eedcf711ef9 service neutron] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:14:30.666567 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96d3c28-490d-4632-b519-6eedcf711ef9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:14:30.666863 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96d3c28-490d-4632-b519-6eedcf711ef9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:14:30.668526 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-c96d3c28-490d-4632-b519-6eedcf711ef9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:14:30.750672 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:14:30.775895 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96d3c28-490d-4632-b519-6eedcf711ef9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:14:30.777417 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96d3c28-490d-4632-b519-6eedcf711ef9 service neutron] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:14:30.796234 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 656/1337] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:14:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:14:33.226584 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaf5749-11fa-4046-a1a5-3d6664ceca5a None None] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:14:33.227114 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaf5749-11fa-4046-a1a5-3d6664ceca5a None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:14:33.227114 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaf5749-11fa-4046-a1a5-3d6664ceca5a None None] PATH_INFO: `/` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:14:33.228482 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 682/1338] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:14:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:14:33.233731 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959bc2ea-0b47-402b-a29a-62ede1b5ed77 None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:14:33.234182 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959bc2ea-0b47-402b-a29a-62ede1b5ed77 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:14:33.234316 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959bc2ea-0b47-402b-a29a-62ede1b5ed77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:14:33.299703 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-959bc2ea-0b47-402b-a29a-62ede1b5ed77 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:14:33.300927 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-959bc2ea-0b47-402b-a29a-62ede1b5ed77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:14:33.394446 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 657/1339] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:14:33 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:14:33.677126 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efaf5749-11fa-4046-a1a5-3d6664ceca5a None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:14:33.678692 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-efaf5749-11fa-4046-a1a5-3d6664ceca5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:14:33.755714 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:14:33.782633 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efaf5749-11fa-4046-a1a5-3d6664ceca5a None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:14:33.853911 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2273923c-2f16-449d-8aaa-5e2cd4d09b1d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0817c3320d5d41d8bba0550c8ae8dd60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:14:33.854511 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2273923c-2f16-449d-8aaa-5e2cd4d09b1d service neutron] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:14:33.854786 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2273923c-2f16-449d-8aaa-5e2cd4d09b1d service neutron] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:14:33.855054 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2273923c-2f16-449d-8aaa-5e2cd4d09b1d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:14:33.856657 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-2273923c-2f16-449d-8aaa-5e2cd4d09b1d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:14:33.966937 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2273923c-2f16-449d-8aaa-5e2cd4d09b1d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:14:33.968371 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2273923c-2f16-449d-8aaa-5e2cd4d09b1d service neutron] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:14:33.985961 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 683/1340] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:14:33 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:14:38.046644 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0f063c-d1ad-4859-8cdd-dc43a588056f None None] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:14:38.047208 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0f063c-d1ad-4859-8cdd-dc43a588056f None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:14:38.047208 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0f063c-d1ad-4859-8cdd-dc43a588056f None None] PATH_INFO: `/` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:14:38.048597 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 658/1341] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:14:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:14:38.053900 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37692ab7-7e23-4831-a36f-5f007dc6de03 None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:14:38.054317 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37692ab7-7e23-4831-a36f-5f007dc6de03 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:14:38.054608 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37692ab7-7e23-4831-a36f-5f007dc6de03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:14:38.060137 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:14:38.120880 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-37692ab7-7e23-4831-a36f-5f007dc6de03 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:14:38.122108 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-37692ab7-7e23-4831-a36f-5f007dc6de03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:14:38.222737 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 684/1342] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:14:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 01:14:38.504005 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0f063c-d1ad-4859-8cdd-dc43a588056f None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:14:38.505695 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-bb0f063c-d1ad-4859-8cdd-dc43a588056f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:14:38.553617 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:14:38.608586 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0f063c-d1ad-4859-8cdd-dc43a588056f None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:14:38.684182 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d92d33-0866-4b02-b74e-2d41fb0db943 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0817c3320d5d41d8bba0550c8ae8dd60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:14:38.684850 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d92d33-0866-4b02-b74e-2d41fb0db943 service neutron] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:14:38.685132 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d92d33-0866-4b02-b74e-2d41fb0db943 service neutron] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:14:38.685407 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d92d33-0866-4b02-b74e-2d41fb0db943 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:14:38.687047 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-67d92d33-0866-4b02-b74e-2d41fb0db943 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:14:38.783535 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:14:38.799850 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d92d33-0866-4b02-b74e-2d41fb0db943 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:14:38.801994 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d92d33-0866-4b02-b74e-2d41fb0db943 service neutron] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:14:38.820499 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 659/1343] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:14:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:14:41.782698 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0d242d-fc91-4e30-9748-941c9044f453 None None] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:14:41.783233 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0d242d-fc91-4e30-9748-941c9044f453 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:14:41.783233 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0d242d-fc91-4e30-9748-941c9044f453 None None] PATH_INFO: `/` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:14:41.784509 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 685/1344] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:14:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:14:41.790043 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7673d8-0f69-4d33-a0c1-9f3c9a76fce4 None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:14:41.790488 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7673d8-0f69-4d33-a0c1-9f3c9a76fce4 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:14:41.790731 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7673d8-0f69-4d33-a0c1-9f3c9a76fce4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:14:41.856659 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-3b7673d8-0f69-4d33-a0c1-9f3c9a76fce4 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:14:41.857878 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-3b7673d8-0f69-4d33-a0c1-9f3c9a76fce4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:14:41.959147 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 660/1345] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:14:41 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:14:42.244600 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0d242d-fc91-4e30-9748-941c9044f453 None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:14:42.246165 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-4c0d242d-fc91-4e30-9748-941c9044f453 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:14:42.333803 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:14:42.351486 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0d242d-fc91-4e30-9748-941c9044f453 None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:14:42.430523 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600e0415-8645-4d3a-80a2-83b2825e72b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0817c3320d5d41d8bba0550c8ae8dd60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:14:42.431157 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600e0415-8645-4d3a-80a2-83b2825e72b0 service neutron] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:14:42.431443 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600e0415-8645-4d3a-80a2-83b2825e72b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:14:42.431719 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600e0415-8645-4d3a-80a2-83b2825e72b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:14:42.433392 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-600e0415-8645-4d3a-80a2-83b2825e72b0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:14:42.546166 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600e0415-8645-4d3a-80a2-83b2825e72b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:14:42.547616 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600e0415-8645-4d3a-80a2-83b2825e72b0 service neutron] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:14:42.565984 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 686/1346] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:14:42 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:14:46.782127 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935917ba-d3b1-4f10-96d3-f9e2a5172757 None None] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:14:46.782711 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935917ba-d3b1-4f10-96d3-f9e2a5172757 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:14:46.782711 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935917ba-d3b1-4f10-96d3-f9e2a5172757 None None] PATH_INFO: `/` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:14:46.784194 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 661/1347] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:14:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:14:46.789995 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9194f7ba-ecc7-4d9d-a4f7-9549799d385a None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:14:46.790481 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9194f7ba-ecc7-4d9d-a4f7-9549799d385a None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:14:46.790583 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9194f7ba-ecc7-4d9d-a4f7-9549799d385a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:14:46.806321 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:14:46.860962 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-9194f7ba-ecc7-4d9d-a4f7-9549799d385a None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:14:46.862191 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-9194f7ba-ecc7-4d9d-a4f7-9549799d385a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:14:46.965429 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 687/1348] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:14:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:14:47.246932 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935917ba-d3b1-4f10-96d3-f9e2a5172757 None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:14:47.249462 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-935917ba-d3b1-4f10-96d3-f9e2a5172757 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:14:47.308705 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:14:47.354573 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935917ba-d3b1-4f10-96d3-f9e2a5172757 None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:14:47.427284 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d89ef8-a79e-423f-8200-d6abd1b600cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0817c3320d5d41d8bba0550c8ae8dd60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:14:47.427887 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d89ef8-a79e-423f-8200-d6abd1b600cf service neutron] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:14:47.428170 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d89ef8-a79e-423f-8200-d6abd1b600cf service neutron] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:14:47.428448 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d89ef8-a79e-423f-8200-d6abd1b600cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:14:47.430072 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-d6d89ef8-a79e-423f-8200-d6abd1b600cf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:14:47.532932 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:14:47.536309 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d89ef8-a79e-423f-8200-d6abd1b600cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:14:47.537901 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d89ef8-a79e-423f-8200-d6abd1b600cf service neutron] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:14:47.555846 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 662/1349] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:14:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:14:52.393700 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832b33a2-ca10-4e0f-bf57-28ad57028b70 None None] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:14:52.395098 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832b33a2-ca10-4e0f-bf57-28ad57028b70 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:14:52.395516 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832b33a2-ca10-4e0f-bf57-28ad57028b70 None None] PATH_INFO: `/` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:14:52.397341 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 688/1350] 10.222.0.20 () {54 vars in 921 bytes} [Wed Aug 31 01:14:52 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:14:52.404083 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9209d888-6dd5-43b7-8879-7756d7659ff2 None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:14:52.404516 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9209d888-6dd5-43b7-8879-7756d7659ff2 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:14:52.404699 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9209d888-6dd5-43b7-8879-7756d7659ff2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:14:52.500054 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-9209d888-6dd5-43b7-8879-7756d7659ff2 None None] MFA Rules not processed for user `dea0fe5b1d6b405cb47b95192580303b`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:14:52.501237 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-9209d888-6dd5-43b7-8879-7756d7659ff2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:14:52.639452 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 663/1351] 10.222.0.20 () {58 vars in 1018 bytes} [Wed Aug 31 01:14:52 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:14:56.833644 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c503170f-89fc-4cc4-8af2-4013bd89ae95 None None] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:14:56.834268 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c503170f-89fc-4cc4-8af2-4013bd89ae95 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:14:56.834268 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c503170f-89fc-4cc4-8af2-4013bd89ae95 None None] PATH_INFO: `/` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:14:56.835575 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 689/1352] 10.222.0.20 () {54 vars in 921 bytes} [Wed Aug 31 01:14:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:14:56.841681 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1affd2-149a-4ad2-906d-44eb4b619f82 None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:14:56.842110 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1affd2-149a-4ad2-906d-44eb4b619f82 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:14:56.842300 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1affd2-149a-4ad2-906d-44eb4b619f82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:14:56.906340 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:14:56.932074 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-fa1affd2-149a-4ad2-906d-44eb4b619f82 None None] MFA Rules not processed for user `dea0fe5b1d6b405cb47b95192580303b`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:14:56.933314 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-fa1affd2-149a-4ad2-906d-44eb4b619f82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:14:56.965877 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=59 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:14:57.045960 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 664/1353] 10.222.0.20 () {58 vars in 1018 bytes} [Wed Aug 31 01:14:56 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:14:58.474305 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aba8f20-d83c-4808-bec5-13a5494d7ef7 None None] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:14:58.474850 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aba8f20-d83c-4808-bec5-13a5494d7ef7 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:14:58.474850 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aba8f20-d83c-4808-bec5-13a5494d7ef7 None None] PATH_INFO: `/` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:14:58.476194 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 690/1354] 10.222.0.20 () {54 vars in 921 bytes} [Wed Aug 31 01:14:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:14:58.481489 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2115e45-7f63-4b38-965a-acc392372bde None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:14:58.481923 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2115e45-7f63-4b38-965a-acc392372bde None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:14:58.482164 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2115e45-7f63-4b38-965a-acc392372bde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:14:58.562539 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-b2115e45-7f63-4b38-965a-acc392372bde None None] MFA Rules not processed for user `dea0fe5b1d6b405cb47b95192580303b`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:14:58.563733 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-b2115e45-7f63-4b38-965a-acc392372bde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:14:58.606004 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:14:58.671545 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 665/1355] 10.222.0.20 () {58 vars in 1018 bytes} [Wed Aug 31 01:14:58 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 01:15:02.005560 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea08ea2e-7b01-4a00-a226-c17761856973 None None] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:15:02.006090 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea08ea2e-7b01-4a00-a226-c17761856973 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:15:02.006208 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea08ea2e-7b01-4a00-a226-c17761856973 None None] PATH_INFO: `/` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:15:02.007499 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 691/1356] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:15:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:15:02.012903 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90489b69-9b46-43d6-9aef-181ddf79cb38 None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:15:02.013304 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90489b69-9b46-43d6-9aef-181ddf79cb38 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:15:02.013641 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90489b69-9b46-43d6-9aef-181ddf79cb38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:15:02.101550 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-90489b69-9b46-43d6-9aef-181ddf79cb38 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:15:02.102042 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-90489b69-9b46-43d6-9aef-181ddf79cb38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:15:02.252776 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 666/1357] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:15:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 01:15:02.561387 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b2ca45-45a9-4896-8c68-c75e6dd0f9c8 None None] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:15:02.561861 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b2ca45-45a9-4896-8c68-c75e6dd0f9c8 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:15:02.562389 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b2ca45-45a9-4896-8c68-c75e6dd0f9c8 None None] PATH_INFO: `/` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:15:02.564153 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 692/1358] 10.222.0.20 () {54 vars in 962 bytes} [Wed Aug 31 01:15:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:15:02.573578 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5559554-8812-4831-b6b6-123a60258f13 None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:15:02.573867 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5559554-8812-4831-b6b6-123a60258f13 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:15:02.574157 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5559554-8812-4831-b6b6-123a60258f13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:15:02.592945 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:15:02.656228 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-b5559554-8812-4831-b6b6-123a60258f13 None None] MFA Rules not processed for user `8744c9de803048f583d0317d01f6a540`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:15:02.657469 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-b5559554-8812-4831-b6b6-123a60258f13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:15:02.795970 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 667/1359] 10.222.0.20 () {58 vars in 1059 bytes} [Wed Aug 31 01:15:02 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:15:02.805257 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b2ca45-45a9-4896-8c68-c75e6dd0f9c8 None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:15:02.806992 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-92b2ca45-45a9-4896-8c68-c75e6dd0f9c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:15:02.975891 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b2ca45-45a9-4896-8c68-c75e6dd0f9c8 None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:15:03.053750 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:15:03.094897 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f523c42-81f1-4051-9ac8-e28f0a86b371 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8744c9de803048f583d0317d01f6a540', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:15:03.095509 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f523c42-81f1-4051-9ac8-e28f0a86b371 service nova] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:15:03.095810 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f523c42-81f1-4051-9ac8-e28f0a86b371 service nova] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:15:03.096068 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f523c42-81f1-4051-9ac8-e28f0a86b371 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:15:03.098224 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-7f523c42-81f1-4051-9ac8-e28f0a86b371 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:15:03.189885 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f523c42-81f1-4051-9ac8-e28f0a86b371 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:15:03.191556 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f523c42-81f1-4051-9ac8-e28f0a86b371 service nova] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:15:03.212834 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 693/1360] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:15:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 01:15:05.358643 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f219093a-1ef1-4efa-befc-dfd12b222848 None None] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:15:05.359198 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f219093a-1ef1-4efa-befc-dfd12b222848 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:15:05.359198 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f219093a-1ef1-4efa-befc-dfd12b222848 None None] PATH_INFO: `/` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:15:05.360552 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 668/1361] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:15:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:15:05.366265 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f8dffc-3f45-485a-a1a7-e31e3707b4d9 None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:15:05.366700 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f8dffc-3f45-485a-a1a7-e31e3707b4d9 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:15:05.366829 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f8dffc-3f45-485a-a1a7-e31e3707b4d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:15:05.436673 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-56f8dffc-3f45-485a-a1a7-e31e3707b4d9 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:15:05.437928 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-56f8dffc-3f45-485a-a1a7-e31e3707b4d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:15:05.478074 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:15:05.543873 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 694/1362] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:15:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:15:05.840203 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f219093a-1ef1-4efa-befc-dfd12b222848 None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:15:05.841869 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-f219093a-1ef1-4efa-befc-dfd12b222848 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:15:05.919627 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:15:06.013139 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f219093a-1ef1-4efa-befc-dfd12b222848 None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:15:06.131079 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a076f237-3ee6-4b92-a047-bf3daa32cffb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8744c9de803048f583d0317d01f6a540', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:15:06.131614 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a076f237-3ee6-4b92-a047-bf3daa32cffb service nova] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:15:06.131878 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a076f237-3ee6-4b92-a047-bf3daa32cffb service nova] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:15:06.132156 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a076f237-3ee6-4b92-a047-bf3daa32cffb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:15:06.133786 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-a076f237-3ee6-4b92-a047-bf3daa32cffb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:15:06.160337 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:15:06.249681 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a076f237-3ee6-4b92-a047-bf3daa32cffb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:15:06.251435 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a076f237-3ee6-4b92-a047-bf3daa32cffb service nova] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:15:06.278313 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 669/1363] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:15:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:15:08.644491 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36e4d9f-dcd5-4d48-875f-fde2e1d0ea27 None None] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:15:08.645644 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36e4d9f-dcd5-4d48-875f-fde2e1d0ea27 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:15:08.646135 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36e4d9f-dcd5-4d48-875f-fde2e1d0ea27 None None] PATH_INFO: `/` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:15:08.651681 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 695/1364] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:15:08 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:15:08.656745 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8b4401-1091-4fcf-957c-f6f4b7879bfb None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:15:08.656745 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8b4401-1091-4fcf-957c-f6f4b7879bfb None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:15:08.656745 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8b4401-1091-4fcf-957c-f6f4b7879bfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:15:08.728222 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-6a8b4401-1091-4fcf-957c-f6f4b7879bfb None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:15:08.729616 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-6a8b4401-1091-4fcf-957c-f6f4b7879bfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:15:08.841366 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:15:08.847722 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 670/1365] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:15:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:15:09.138922 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7516a0f9-7f86-4d27-9681-921a9f453a85 None None] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:15:09.139343 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7516a0f9-7f86-4d27-9681-921a9f453a85 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:15:09.139720 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7516a0f9-7f86-4d27-9681-921a9f453a85 None None] PATH_INFO: `/` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:15:09.141364 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 696/1366] 10.222.0.20 () {54 vars in 962 bytes} [Wed Aug 31 01:15:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:15:09.147699 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3179d3e-6cf9-49a2-89d8-96a35b17778a None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:15:09.147992 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3179d3e-6cf9-49a2-89d8-96a35b17778a None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:15:09.150766 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3179d3e-6cf9-49a2-89d8-96a35b17778a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:15:09.228864 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-a3179d3e-6cf9-49a2-89d8-96a35b17778a None None] MFA Rules not processed for user `8744c9de803048f583d0317d01f6a540`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:15:09.230086 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-a3179d3e-6cf9-49a2-89d8-96a35b17778a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:15:09.356060 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 671/1367] 10.222.0.20 () {58 vars in 1059 bytes} [Wed Aug 31 01:15:09 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:15:09.365690 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7516a0f9-7f86-4d27-9681-921a9f453a85 None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:15:09.367725 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-7516a0f9-7f86-4d27-9681-921a9f453a85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:15:09.523429 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7516a0f9-7f86-4d27-9681-921a9f453a85 None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:15:09.544209 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:15:09.643592 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5f7653-8290-4784-8ea0-bf032f1e9001 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8744c9de803048f583d0317d01f6a540', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:15:09.644988 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5f7653-8290-4784-8ea0-bf032f1e9001 service nova] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:15:09.645545 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5f7653-8290-4784-8ea0-bf032f1e9001 service nova] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:15:09.645692 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5f7653-8290-4784-8ea0-bf032f1e9001 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:15:09.647438 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-bc5f7653-8290-4784-8ea0-bf032f1e9001 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:15:09.750957 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5f7653-8290-4784-8ea0-bf032f1e9001 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:15:09.752644 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5f7653-8290-4784-8ea0-bf032f1e9001 service nova] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:15:09.772904 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 697/1368] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:15:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:15:10.215434 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c46390d-8512-4b47-ad21-0314fbcd5b2f None None] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:15:10.216015 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c46390d-8512-4b47-ad21-0314fbcd5b2f None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:15:10.216015 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c46390d-8512-4b47-ad21-0314fbcd5b2f None None] PATH_INFO: `/` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:15:10.217672 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 672/1369] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:15:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:15:10.223452 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49541eb-ca57-41b7-9f99-b13b8290babc None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:15:10.223744 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49541eb-ca57-41b7-9f99-b13b8290babc None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:15:10.224021 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49541eb-ca57-41b7-9f99-b13b8290babc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:15:10.257712 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:15:10.295801 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-d49541eb-ca57-41b7-9f99-b13b8290babc None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:15:10.296969 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-d49541eb-ca57-41b7-9f99-b13b8290babc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:15:10.403438 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 698/1370] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:15:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:15:10.989231 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b897e4c-4535-4928-830d-850f8bf335a1 None None] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:15:10.989231 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b897e4c-4535-4928-830d-850f8bf335a1 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:15:10.989231 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b897e4c-4535-4928-830d-850f8bf335a1 None None] PATH_INFO: `/` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:15:10.991571 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 673/1371] 10.222.0.20 () {54 vars in 964 bytes} [Wed Aug 31 01:15:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:15:10.994804 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b12ec6f-92ad-4656-9e74-108acae91338 None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:15:10.995298 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b12ec6f-92ad-4656-9e74-108acae91338 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:15:10.995298 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b12ec6f-92ad-4656-9e74-108acae91338 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:15:11.075896 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-6b12ec6f-92ad-4656-9e74-108acae91338 None None] MFA Rules not processed for user `a5c8fc69c518412ba1aad61a9f4f558c`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:15:11.077165 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-6b12ec6f-92ad-4656-9e74-108acae91338 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:15:11.125960 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:15:11.157730 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 699/1372] 10.222.0.20 () {58 vars in 1061 bytes} [Wed Aug 31 01:15:10 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:15:11.165347 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b897e4c-4535-4928-830d-850f8bf335a1 None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:15:11.167164 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-5b897e4c-4535-4928-830d-850f8bf335a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:15:11.217916 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:15:11.272853 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b897e4c-4535-4928-830d-850f8bf335a1 None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:15:11.350625 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dae80a8-c71a-4aa9-83a8-3bda8fa828f6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5c8fc69c518412ba1aad61a9f4f558c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:15:11.351238 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dae80a8-c71a-4aa9-83a8-3bda8fa828f6 service cinder] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:15:11.351510 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dae80a8-c71a-4aa9-83a8-3bda8fa828f6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:15:11.351781 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dae80a8-c71a-4aa9-83a8-3bda8fa828f6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:15:11.353389 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-5dae80a8-c71a-4aa9-83a8-3bda8fa828f6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:15:11.468379 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:15:11.481758 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dae80a8-c71a-4aa9-83a8-3bda8fa828f6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:15:11.483210 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dae80a8-c71a-4aa9-83a8-3bda8fa828f6 service cinder] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:15:11.502014 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 674/1373] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:15:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:15:11.908653 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336f3667-5fd5-4b32-a004-3458554df890 None None] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:15:11.908914 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336f3667-5fd5-4b32-a004-3458554df890 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:15:11.909720 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336f3667-5fd5-4b32-a004-3458554df890 None None] PATH_INFO: `/` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:15:11.911073 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 700/1374] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:15:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:15:11.916136 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b65718d-2999-4b1f-a361-6ece4243fefb None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:15:11.916430 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b65718d-2999-4b1f-a361-6ece4243fefb None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:15:11.916710 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b65718d-2999-4b1f-a361-6ece4243fefb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:15:11.982845 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-2b65718d-2999-4b1f-a361-6ece4243fefb None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:15:11.984066 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-2b65718d-2999-4b1f-a361-6ece4243fefb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:15:12.080582 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 675/1375] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:15:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:15:12.373882 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336f3667-5fd5-4b32-a004-3458554df890 None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:15:12.375492 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-336f3667-5fd5-4b32-a004-3458554df890 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:15:12.520449 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336f3667-5fd5-4b32-a004-3458554df890 None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:15:12.570284 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:15:12.631755 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1519db-a633-44a0-80d8-011181d1eece service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8744c9de803048f583d0317d01f6a540', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:15:12.632426 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1519db-a633-44a0-80d8-011181d1eece service nova] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:15:12.632758 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1519db-a633-44a0-80d8-011181d1eece service nova] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:15:12.633024 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1519db-a633-44a0-80d8-011181d1eece service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:15:12.634637 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-5b1519db-a633-44a0-80d8-011181d1eece service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:15:12.727495 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1519db-a633-44a0-80d8-011181d1eece service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:15:12.729128 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1519db-a633-44a0-80d8-011181d1eece service nova] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:15:12.747917 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 701/1376] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:15:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:15:14.830184 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425f0d57-5f8f-4e46-9550-a612f133823e None None] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:15:14.830676 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425f0d57-5f8f-4e46-9550-a612f133823e None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:15:14.830676 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425f0d57-5f8f-4e46-9550-a612f133823e None None] PATH_INFO: `/` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:15:14.832664 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 676/1377] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:15:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:15:14.837834 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfcbdfe-a422-4283-b97c-bd6cba5953d3 None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:15:14.838259 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfcbdfe-a422-4283-b97c-bd6cba5953d3 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:15:14.838518 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfcbdfe-a422-4283-b97c-bd6cba5953d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:15:14.911357 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-ebfcbdfe-a422-4283-b97c-bd6cba5953d3 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:15:14.912583 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-ebfcbdfe-a422-4283-b97c-bd6cba5953d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:15:14.919177 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:15:15.024945 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 702/1378] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:15:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:15:15.318701 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425f0d57-5f8f-4e46-9550-a612f133823e None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:15:15.321703 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-425f0d57-5f8f-4e46-9550-a612f133823e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:15:15.391241 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:15:15.483648 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425f0d57-5f8f-4e46-9550-a612f133823e None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:15:15.607701 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21182ef2-6c4c-45b6-99c2-a5da00320ed7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8744c9de803048f583d0317d01f6a540', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:15:15.608230 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21182ef2-6c4c-45b6-99c2-a5da00320ed7 service nova] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:15:15.608509 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21182ef2-6c4c-45b6-99c2-a5da00320ed7 service nova] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:15:15.608817 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21182ef2-6c4c-45b6-99c2-a5da00320ed7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:15:15.610551 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-21182ef2-6c4c-45b6-99c2-a5da00320ed7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:15:15.631104 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:15:15.710479 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21182ef2-6c4c-45b6-99c2-a5da00320ed7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:15:15.712220 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21182ef2-6c4c-45b6-99c2-a5da00320ed7 service nova] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:15:15.732530 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 677/1379] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:15:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:15:17.820787 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0813eccc-7dc8-4812-be19-9883edf9da17 None None] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:15:17.821379 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0813eccc-7dc8-4812-be19-9883edf9da17 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:15:17.821379 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0813eccc-7dc8-4812-be19-9883edf9da17 None None] PATH_INFO: `/` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:15:17.822842 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 703/1380] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:15:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:15:17.827855 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f7c23d-825f-49d5-91cc-80c9ce4963e3 None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:15:17.828270 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f7c23d-825f-49d5-91cc-80c9ce4963e3 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:15:17.828479 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f7c23d-825f-49d5-91cc-80c9ce4963e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:15:17.900844 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-a8f7c23d-825f-49d5-91cc-80c9ce4963e3 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:15:17.901906 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-a8f7c23d-825f-49d5-91cc-80c9ce4963e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:15:18.021472 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:15:18.028482 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 678/1381] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:15:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:15:18.319649 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0813eccc-7dc8-4812-be19-9883edf9da17 None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:15:18.321193 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-0813eccc-7dc8-4812-be19-9883edf9da17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:15:18.476825 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:15:18.487910 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0813eccc-7dc8-4812-be19-9883edf9da17 None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:15:18.611456 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8446b9a5-a0e2-47a6-bb7d-2ac76aa2c9eb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8744c9de803048f583d0317d01f6a540', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:15:18.612039 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8446b9a5-a0e2-47a6-bb7d-2ac76aa2c9eb service nova] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:15:18.612837 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8446b9a5-a0e2-47a6-bb7d-2ac76aa2c9eb service nova] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:15:18.613115 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8446b9a5-a0e2-47a6-bb7d-2ac76aa2c9eb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:15:18.614934 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-8446b9a5-a0e2-47a6-bb7d-2ac76aa2c9eb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:15:18.714743 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8446b9a5-a0e2-47a6-bb7d-2ac76aa2c9eb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:15:18.716358 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8446b9a5-a0e2-47a6-bb7d-2ac76aa2c9eb service nova] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:15:18.729043 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:15:18.739183 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 704/1382] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:15:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:15:20.978189 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7019f24e-25ba-46b4-8185-baf34e094c05 None None] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:15:20.978750 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7019f24e-25ba-46b4-8185-baf34e094c05 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:15:20.978911 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7019f24e-25ba-46b4-8185-baf34e094c05 None None] PATH_INFO: `/` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:15:20.980923 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 679/1383] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:15:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:15:20.985882 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35943c8-f807-4722-a7f7-6a6f5e9003fc None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:15:20.986279 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35943c8-f807-4722-a7f7-6a6f5e9003fc None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:15:20.986442 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35943c8-f807-4722-a7f7-6a6f5e9003fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:15:21.055325 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-d35943c8-f807-4722-a7f7-6a6f5e9003fc None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:15:21.056549 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-d35943c8-f807-4722-a7f7-6a6f5e9003fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:15:21.155290 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 705/1384] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:15:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:15:21.442768 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7019f24e-25ba-46b4-8185-baf34e094c05 None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:15:21.444832 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-7019f24e-25ba-46b4-8185-baf34e094c05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:15:21.612874 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7019f24e-25ba-46b4-8185-baf34e094c05 None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:15:21.696559 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:15:21.755695 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e305fcb2-9388-4660-93b2-faa60495e68e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8744c9de803048f583d0317d01f6a540', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:15:21.756467 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e305fcb2-9388-4660-93b2-faa60495e68e service nova] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:15:21.756865 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e305fcb2-9388-4660-93b2-faa60495e68e service nova] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:15:21.757215 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e305fcb2-9388-4660-93b2-faa60495e68e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:15:21.759562 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-e305fcb2-9388-4660-93b2-faa60495e68e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:15:21.859297 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e305fcb2-9388-4660-93b2-faa60495e68e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:15:21.860957 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e305fcb2-9388-4660-93b2-faa60495e68e service nova] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:15:21.880749 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 680/1385] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:15:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:15:23.796516 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb407e9-0423-4786-ac82-8288d92ed3e8 None None] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:15:23.797294 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb407e9-0423-4786-ac82-8288d92ed3e8 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:15:23.797294 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb407e9-0423-4786-ac82-8288d92ed3e8 None None] PATH_INFO: `/` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:15:23.798515 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 706/1386] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:15:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:15:23.804201 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddea806-3492-4523-bb51-4258866a9a90 None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:15:23.804752 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddea806-3492-4523-bb51-4258866a9a90 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:15:23.804858 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddea806-3492-4523-bb51-4258866a9a90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:15:23.873045 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-2ddea806-3492-4523-bb51-4258866a9a90 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:15:23.874253 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-2ddea806-3492-4523-bb51-4258866a9a90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:15:23.909069 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:15:23.976544 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 681/1387] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:15:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:15:24.268849 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb407e9-0423-4786-ac82-8288d92ed3e8 None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:15:24.270372 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-9fb407e9-0423-4786-ac82-8288d92ed3e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:15:24.348313 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:15:24.425262 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb407e9-0423-4786-ac82-8288d92ed3e8 None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:15:24.541179 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d74838-3536-4045-9040-9e34699251ba service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8744c9de803048f583d0317d01f6a540', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:15:24.541791 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d74838-3536-4045-9040-9e34699251ba service nova] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:15:24.542077 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d74838-3536-4045-9040-9e34699251ba service nova] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:15:24.542356 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d74838-3536-4045-9040-9e34699251ba service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:15:24.543980 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-49d74838-3536-4045-9040-9e34699251ba service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:15:24.582710 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:15:24.639720 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d74838-3536-4045-9040-9e34699251ba service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:15:24.641387 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d74838-3536-4045-9040-9e34699251ba service nova] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:15:24.659952 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 707/1388] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:15:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:15:26.711324 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983d3038-72b3-4e94-a359-dc576d7071f4 None None] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:15:26.711906 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983d3038-72b3-4e94-a359-dc576d7071f4 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:15:26.711906 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983d3038-72b3-4e94-a359-dc576d7071f4 None None] PATH_INFO: `/` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:15:26.713258 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 682/1389] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:15:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:15:26.718432 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b91b66e-d296-40d8-8eed-17d7f057ff33 None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:15:26.718886 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b91b66e-d296-40d8-8eed-17d7f057ff33 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:15:26.718990 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b91b66e-d296-40d8-8eed-17d7f057ff33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:15:26.735561 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2824b0d-8ae1-4a1a-8605-c368382a59d6 None None] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:15:26.735844 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2824b0d-8ae1-4a1a-8605-c368382a59d6 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:15:26.736117 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2824b0d-8ae1-4a1a-8605-c368382a59d6 None None] PATH_INFO: `/` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:15:26.737511 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 683/1390] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:15:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 01:15:26.742548 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff9641e-1dbf-4fa2-987b-bba5e360d311 None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:15:26.742836 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff9641e-1dbf-4fa2-987b-bba5e360d311 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:15:26.743118 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff9641e-1dbf-4fa2-987b-bba5e360d311 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:15:26.785693 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-4b91b66e-d296-40d8-8eed-17d7f057ff33 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:15:26.786827 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-4b91b66e-d296-40d8-8eed-17d7f057ff33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:15:26.809779 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-4ff9641e-1dbf-4fa2-987b-bba5e360d311 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:15:26.810966 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-4ff9641e-1dbf-4fa2-987b-bba5e360d311 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:15:26.882871 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 708/1391] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:15:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:15:26.909994 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 684/1392] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:15:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 01:15:27.167941 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b91b66e-d296-40d8-8eed-17d7f057ff33 None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:15:27.169260 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-4b91b66e-d296-40d8-8eed-17d7f057ff33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:15:27.183821 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:15:27.214633 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5366f7-b628-447a-be78-33668cbc6c3b None None] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:15:27.214998 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5366f7-b628-447a-be78-33668cbc6c3b None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:15:27.215347 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5366f7-b628-447a-be78-33668cbc6c3b None None] PATH_INFO: `/` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:15:27.216815 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 685/1393] 10.222.0.20 () {54 vars in 967 bytes} [Wed Aug 31 01:15:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:15:27.221620 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed8030d-8bef-444e-a234-4a64dcbe8a98 None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:15:27.221970 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed8030d-8bef-444e-a234-4a64dcbe8a98 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:15:27.222362 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed8030d-8bef-444e-a234-4a64dcbe8a98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:15:27.238491 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:15:27.285513 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b91b66e-d296-40d8-8eed-17d7f057ff33 None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:15:27.298723 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-bed8030d-8bef-444e-a234-4a64dcbe8a98 None None] MFA Rules not processed for user `d8590c288028418ebe7a4163fd8039f1`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:15:27.300551 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-bed8030d-8bef-444e-a234-4a64dcbe8a98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:15:27.378110 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 686/1394] 10.222.0.20 () {58 vars in 1064 bytes} [Wed Aug 31 01:15:27 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 01:15:27.385449 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed8030d-8bef-444e-a234-4a64dcbe8a98 None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:15:27.393550 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed8030d-8bef-444e-a234-4a64dcbe8a98 None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:15:27.403472 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e7ca7b-4c90-4b41-b093-f458fdf149c9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8744c9de803048f583d0317d01f6a540', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:15:27.404085 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e7ca7b-4c90-4b41-b093-f458fdf149c9 service nova] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:15:27.404380 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e7ca7b-4c90-4b41-b093-f458fdf149c9 service nova] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:15:27.404635 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e7ca7b-4c90-4b41-b093-f458fdf149c9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:15:27.406437 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-27e7ca7b-4c90-4b41-b093-f458fdf149c9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:15:27.415723 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:15:27.469533 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc16c20-866e-475d-a8d7-aa1d8b94073a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8590c288028418ebe7a4163fd8039f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:15:27.470139 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc16c20-866e-475d-a8d7-aa1d8b94073a service glance] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:15:27.470475 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc16c20-866e-475d-a8d7-aa1d8b94073a service glance] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:15:27.470888 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc16c20-866e-475d-a8d7-aa1d8b94073a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:15:27.472540 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-7dc16c20-866e-475d-a8d7-aa1d8b94073a service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:15:27.497581 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e7ca7b-4c90-4b41-b093-f458fdf149c9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:15:27.499196 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e7ca7b-4c90-4b41-b093-f458fdf149c9 service nova] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:15:27.502878 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:15:27.517803 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 709/1395] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:15:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:15:27.583037 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc16c20-866e-475d-a8d7-aa1d8b94073a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:15:27.584484 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc16c20-866e-475d-a8d7-aa1d8b94073a service glance] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:15:27.608853 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 687/1396] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:15:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 01:15:28.474317 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41e3c81-5362-4f30-a727-1ff6364d8ac9 None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:15:28.474865 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41e3c81-5362-4f30-a727-1ff6364d8ac9 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:15:28.474865 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41e3c81-5362-4f30-a727-1ff6364d8ac9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:15:28.547305 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-b41e3c81-5362-4f30-a727-1ff6364d8ac9 None None] MFA Rules not processed for user `b3efb4f9daac4a0ca05a36dd361f17e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:15:28.548505 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-b41e3c81-5362-4f30-a727-1ff6364d8ac9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:15:28.631415 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:15:28.635089 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 710/1397] 10.222.0.20 () {58 vars in 1009 bytes} [Wed Aug 31 01:15:28 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:15:28.682043 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41c48e7-7081-447e-af3d-edb59545ba5b None None] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:15:28.682527 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41c48e7-7081-447e-af3d-edb59545ba5b None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:15:28.682618 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41c48e7-7081-447e-af3d-edb59545ba5b None None] PATH_INFO: `/` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:15:28.683947 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 688/1398] 10.222.0.20 () {54 vars in 943 bytes} [Wed Aug 31 01:15:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:15:28.690371 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03e470f-1fee-4c43-a871-005d1c1414b6 None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:15:28.690736 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03e470f-1fee-4c43-a871-005d1c1414b6 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:15:28.691020 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03e470f-1fee-4c43-a871-005d1c1414b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:15:28.771400 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-c03e470f-1fee-4c43-a871-005d1c1414b6 None None] MFA Rules not processed for user `fc1d010919ec40878ae829cc8dceec4c`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:15:28.772638 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-c03e470f-1fee-4c43-a871-005d1c1414b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:15:28.880539 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 711/1399] 10.222.0.20 () {58 vars in 1040 bytes} [Wed Aug 31 01:15:28 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:15:28.889910 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41c48e7-7081-447e-af3d-edb59545ba5b None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:15:28.891486 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-e41c48e7-7081-447e-af3d-edb59545ba5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:15:29.024680 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:15:29.027316 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41c48e7-7081-447e-af3d-edb59545ba5b None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:15:29.131006 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f769017-f7c6-42fd-9768-a035fc440013 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc1d010919ec40878ae829cc8dceec4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:15:29.131625 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f769017-f7c6-42fd-9768-a035fc440013 service swift] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:15:29.131901 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f769017-f7c6-42fd-9768-a035fc440013 service swift] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:15:29.132170 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f769017-f7c6-42fd-9768-a035fc440013 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:15:29.133943 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-9f769017-f7c6-42fd-9768-a035fc440013 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:15:29.218341 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f769017-f7c6-42fd-9768-a035fc440013 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:15:29.219932 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f769017-f7c6-42fd-9768-a035fc440013 service swift] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:15:29.228272 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 689/1400] 10.222.0.20 () {60 vars in 1384 bytes} [Wed Aug 31 01:15:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 01:15:29.596256 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935a0a71-a471-404f-85b4-1426dd34ac2a None None] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:15:29.596749 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935a0a71-a471-404f-85b4-1426dd34ac2a None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:15:29.596885 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935a0a71-a471-404f-85b4-1426dd34ac2a None None] PATH_INFO: `/` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:15:29.598240 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 712/1401] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:15:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:15:29.606105 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb573850-2482-46e1-b617-db63ba836ac9 None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:15:29.606402 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb573850-2482-46e1-b617-db63ba836ac9 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:15:29.606688 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb573850-2482-46e1-b617-db63ba836ac9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:15:29.649844 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:15:29.684512 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-eb573850-2482-46e1-b617-db63ba836ac9 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:15:29.685804 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-eb573850-2482-46e1-b617-db63ba836ac9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:15:29.793583 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 690/1402] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:15:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:15:30.085567 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935a0a71-a471-404f-85b4-1426dd34ac2a None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:15:30.087179 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-935a0a71-a471-404f-85b4-1426dd34ac2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:15:30.155775 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:15:30.250150 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935a0a71-a471-404f-85b4-1426dd34ac2a None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:15:30.377751 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9285e760-391e-4636-8ebb-b1f26160090c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8744c9de803048f583d0317d01f6a540', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:15:30.378333 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9285e760-391e-4636-8ebb-b1f26160090c service nova] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:15:30.378621 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9285e760-391e-4636-8ebb-b1f26160090c service nova] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:15:30.378897 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9285e760-391e-4636-8ebb-b1f26160090c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:15:30.381340 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-9285e760-391e-4636-8ebb-b1f26160090c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:15:30.400993 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:15:30.481136 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9285e760-391e-4636-8ebb-b1f26160090c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:15:30.482752 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9285e760-391e-4636-8ebb-b1f26160090c service nova] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:15:30.503829 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 713/1403] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:15:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:15:32.656944 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6844620a-86f3-404b-877b-7a8c7eefd48f None None] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:15:32.657623 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6844620a-86f3-404b-877b-7a8c7eefd48f None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:15:32.657623 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6844620a-86f3-404b-877b-7a8c7eefd48f None None] PATH_INFO: `/` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:15:32.659408 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 691/1404] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:15:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:15:32.665234 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776dc4e0-18f3-4e01-a349-233f66688c32 None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:15:32.665848 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776dc4e0-18f3-4e01-a349-233f66688c32 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:15:32.666156 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776dc4e0-18f3-4e01-a349-233f66688c32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:15:32.739730 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-776dc4e0-18f3-4e01-a349-233f66688c32 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:15:32.740974 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-776dc4e0-18f3-4e01-a349-233f66688c32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:15:32.844043 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:15:32.854494 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 714/1405] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:15:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:15:33.153914 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6844620a-86f3-404b-877b-7a8c7eefd48f None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:15:33.155782 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-6844620a-86f3-404b-877b-7a8c7eefd48f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:15:33.273791 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:15:33.322210 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6844620a-86f3-404b-877b-7a8c7eefd48f None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:15:33.453100 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca40340c-f600-4c7c-b2ab-e7ba17953490 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8744c9de803048f583d0317d01f6a540', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:15:33.454235 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca40340c-f600-4c7c-b2ab-e7ba17953490 service nova] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:15:33.455270 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca40340c-f600-4c7c-b2ab-e7ba17953490 service nova] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:15:33.455726 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca40340c-f600-4c7c-b2ab-e7ba17953490 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:15:33.458017 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-ca40340c-f600-4c7c-b2ab-e7ba17953490 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:15:33.545881 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:15:33.566828 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca40340c-f600-4c7c-b2ab-e7ba17953490 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:15:33.568482 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca40340c-f600-4c7c-b2ab-e7ba17953490 service nova] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:15:33.588419 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 692/1406] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:15:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:15:35.674474 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3130945-2479-444a-8eb2-8e15eb013ae2 None None] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:15:35.675032 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3130945-2479-444a-8eb2-8e15eb013ae2 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:15:35.675032 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3130945-2479-444a-8eb2-8e15eb013ae2 None None] PATH_INFO: `/` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:15:35.676411 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 715/1407] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:15:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:15:35.681701 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b9eb16-86ae-41f8-9e2e-7d95c4039422 None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:15:35.682119 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b9eb16-86ae-41f8-9e2e-7d95c4039422 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:15:35.682326 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b9eb16-86ae-41f8-9e2e-7d95c4039422 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:15:35.752745 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-34b9eb16-86ae-41f8-9e2e-7d95c4039422 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:15:35.753983 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-34b9eb16-86ae-41f8-9e2e-7d95c4039422 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:15:35.864812 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 693/1408] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:15:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:15:36.146688 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3130945-2479-444a-8eb2-8e15eb013ae2 None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:15:36.149462 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-f3130945-2479-444a-8eb2-8e15eb013ae2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:15:36.353156 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3130945-2479-444a-8eb2-8e15eb013ae2 None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:15:36.435718 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:15:36.492194 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1580be-c286-43a5-a121-40d3219ab593 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8744c9de803048f583d0317d01f6a540', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:15:36.492752 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1580be-c286-43a5-a121-40d3219ab593 service nova] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:15:36.493193 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1580be-c286-43a5-a121-40d3219ab593 service nova] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:15:36.493479 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1580be-c286-43a5-a121-40d3219ab593 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:15:36.495339 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-bd1580be-c286-43a5-a121-40d3219ab593 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:15:36.597278 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1580be-c286-43a5-a121-40d3219ab593 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:15:36.599156 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1580be-c286-43a5-a121-40d3219ab593 service nova] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:15:36.618731 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 716/1409] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:15:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 473 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:15:45.854682 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=79 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:15:46.616210 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=64 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:15:52.697432 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681e2266-73b1-4e36-99e1-77f828f416de None None] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:15:52.698034 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681e2266-73b1-4e36-99e1-77f828f416de None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:15:52.698034 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681e2266-73b1-4e36-99e1-77f828f416de None None] PATH_INFO: `/` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:15:52.699331 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 694/1410] 10.222.0.20 () {54 vars in 918 bytes} [Wed Aug 31 01:15:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:15:52.704364 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e416b7-5dac-4f55-a32d-a24e0e19ea2b None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:15:52.705130 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e416b7-5dac-4f55-a32d-a24e0e19ea2b None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:15:52.705280 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e416b7-5dac-4f55-a32d-a24e0e19ea2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:15:52.781210 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-18e416b7-5dac-4f55-a32d-a24e0e19ea2b None None] MFA Rules not processed for user `dea0fe5b1d6b405cb47b95192580303b`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:15:52.782426 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-18e416b7-5dac-4f55-a32d-a24e0e19ea2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:15:52.887858 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 717/1411] 10.222.0.20 () {58 vars in 1015 bytes} [Wed Aug 31 01:15:52 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:15:52.902580 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017a72e0-b1de-4d21-8755-0abc6504d737 None None] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:15:52.902874 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017a72e0-b1de-4d21-8755-0abc6504d737 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:15:52.903155 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017a72e0-b1de-4d21-8755-0abc6504d737 None None] PATH_INFO: `/` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:15:52.904494 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 695/1412] 10.222.0.20 () {54 vars in 918 bytes} [Wed Aug 31 01:15:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:15:52.909702 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906852ca-be51-4494-a9ee-cd37ae707339 None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:15:52.909990 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906852ca-be51-4494-a9ee-cd37ae707339 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:15:52.910259 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906852ca-be51-4494-a9ee-cd37ae707339 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:15:52.982186 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:15:52.985473 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-906852ca-be51-4494-a9ee-cd37ae707339 None None] MFA Rules not processed for user `8744c9de803048f583d0317d01f6a540`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:15:52.987319 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-906852ca-be51-4494-a9ee-cd37ae707339 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:15:53.100564 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 718/1413] 10.222.0.20 () {58 vars in 1015 bytes} [Wed Aug 31 01:15:52 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:15:55.476917 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1676b092-2353-4b3a-a202-40fc94bb9aa5 None None] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:15:55.477465 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1676b092-2353-4b3a-a202-40fc94bb9aa5 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:15:55.477465 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1676b092-2353-4b3a-a202-40fc94bb9aa5 None None] PATH_INFO: `/` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:15:55.478872 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 696/1414] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 01:15:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:15:55.483984 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9be9be2-21ac-4907-9cfd-69a74b7a8a01 None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:15:55.484395 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9be9be2-21ac-4907-9cfd-69a74b7a8a01 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:15:55.484532 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9be9be2-21ac-4907-9cfd-69a74b7a8a01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:15:55.549601 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-d9be9be2-21ac-4907-9cfd-69a74b7a8a01 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:15:55.550830 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-d9be9be2-21ac-4907-9cfd-69a74b7a8a01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:15:55.604923 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:15:55.646866 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 719/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 01:15:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:15:55.936502 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1676b092-2353-4b3a-a202-40fc94bb9aa5 None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:15:55.938047 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-1676b092-2353-4b3a-a202-40fc94bb9aa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:15:56.082218 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1676b092-2353-4b3a-a202-40fc94bb9aa5 None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:15:56.153536 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:15:56.193533 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f3cf4f-df20-4c03-a871-e81a4b62b7a5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8744c9de803048f583d0317d01f6a540', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:15:56.194138 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f3cf4f-df20-4c03-a871-e81a4b62b7a5 service nova] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:15:56.194445 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f3cf4f-df20-4c03-a871-e81a4b62b7a5 service nova] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:15:56.194719 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f3cf4f-df20-4c03-a871-e81a4b62b7a5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:15:56.196326 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-d9f3cf4f-df20-4c03-a871-e81a4b62b7a5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:15:56.283215 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f3cf4f-df20-4c03-a871-e81a4b62b7a5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:15:56.284833 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f3cf4f-df20-4c03-a871-e81a4b62b7a5 service nova] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:15:56.302557 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 697/1416] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:15:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:15:57.194101 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c75c4b-0417-46ac-aea7-4b8eda9ed607 None None] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:15:57.194647 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c75c4b-0417-46ac-aea7-4b8eda9ed607 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:15:57.194752 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c75c4b-0417-46ac-aea7-4b8eda9ed607 None None] PATH_INFO: `/` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:15:57.196045 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 720/1417] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 31 01:15:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:15:57.201065 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51268e94-24a9-4b5a-a8b2-06c0179c6d49 None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:15:57.201473 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51268e94-24a9-4b5a-a8b2-06c0179c6d49 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:15:57.201626 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51268e94-24a9-4b5a-a8b2-06c0179c6d49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:15:57.276804 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-51268e94-24a9-4b5a-a8b2-06c0179c6d49 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:15:57.278007 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-51268e94-24a9-4b5a-a8b2-06c0179c6d49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:15:57.304548 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:15:57.373465 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 698/1418] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 31 01:15:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:15:57.523565 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c75c4b-0417-46ac-aea7-4b8eda9ed607 None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:15:57.525080 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-b8c75c4b-0417-46ac-aea7-4b8eda9ed607 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:15:57.667431 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c75c4b-0417-46ac-aea7-4b8eda9ed607 None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:15:57.710355 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:15:57.781713 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39e89b2-5472-4b1f-9358-7fdecf5ce25f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8744c9de803048f583d0317d01f6a540', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:15:57.782252 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39e89b2-5472-4b1f-9358-7fdecf5ce25f service nova] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:15:57.782531 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39e89b2-5472-4b1f-9358-7fdecf5ce25f service nova] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:15:57.782803 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39e89b2-5472-4b1f-9358-7fdecf5ce25f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:15:57.784400 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-d39e89b2-5472-4b1f-9358-7fdecf5ce25f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:15:57.871417 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39e89b2-5472-4b1f-9358-7fdecf5ce25f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:15:57.873052 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39e89b2-5472-4b1f-9358-7fdecf5ce25f service nova] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:15:57.891182 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 721/1419] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:15:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:15:58.785099 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42379d9c-0d75-47de-b851-681d1e17b189 None None] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:15:58.785671 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42379d9c-0d75-47de-b851-681d1e17b189 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:15:58.785671 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42379d9c-0d75-47de-b851-681d1e17b189 None None] PATH_INFO: `/` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:15:58.787022 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 699/1420] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 31 01:15:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:15:58.791971 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fab11e-7247-4497-8394-60d5f21f6b0b None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:15:58.792351 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fab11e-7247-4497-8394-60d5f21f6b0b None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:15:58.792532 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fab11e-7247-4497-8394-60d5f21f6b0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:15:58.847291 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:15:58.869118 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-47fab11e-7247-4497-8394-60d5f21f6b0b None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:15:58.870333 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-47fab11e-7247-4497-8394-60d5f21f6b0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:15:58.965432 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 722/1421] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 31 01:15:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:15:59.115730 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42379d9c-0d75-47de-b851-681d1e17b189 None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:15:59.117293 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-42379d9c-0d75-47de-b851-681d1e17b189 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:15:59.262186 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42379d9c-0d75-47de-b851-681d1e17b189 None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:15:59.274591 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:15:59.375369 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13ba00e-e444-450b-981d-0e70f746d949 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8744c9de803048f583d0317d01f6a540', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:15:59.375897 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13ba00e-e444-450b-981d-0e70f746d949 service nova] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:15:59.376177 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13ba00e-e444-450b-981d-0e70f746d949 service nova] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:15:59.376448 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13ba00e-e444-450b-981d-0e70f746d949 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:15:59.378097 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-c13ba00e-e444-450b-981d-0e70f746d949 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:15:59.465759 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13ba00e-e444-450b-981d-0e70f746d949 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:15:59.467359 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13ba00e-e444-450b-981d-0e70f746d949 service nova] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:15:59.485497 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 700/1422] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:15:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:16:00.336410 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854ed3a3-cc0d-4bc3-8760-b36ebfe5d6b9 None None] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:16:00.337039 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854ed3a3-cc0d-4bc3-8760-b36ebfe5d6b9 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:16:00.337039 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854ed3a3-cc0d-4bc3-8760-b36ebfe5d6b9 None None] PATH_INFO: `/` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:16:00.338341 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 723/1423] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 31 01:16:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:16:00.343343 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4514bb-6408-4f52-a0f6-6f792cd0355f None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:16:00.343779 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4514bb-6408-4f52-a0f6-6f792cd0355f None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:16:00.343935 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4514bb-6408-4f52-a0f6-6f792cd0355f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:16:00.364575 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:16:00.420739 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-da4514bb-6408-4f52-a0f6-6f792cd0355f None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:16:00.421959 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-da4514bb-6408-4f52-a0f6-6f792cd0355f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:16:00.518021 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 701/1424] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 31 01:16:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:16:00.668960 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854ed3a3-cc0d-4bc3-8760-b36ebfe5d6b9 None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:16:00.670495 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-854ed3a3-cc0d-4bc3-8760-b36ebfe5d6b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:16:00.778940 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:16:00.814683 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854ed3a3-cc0d-4bc3-8760-b36ebfe5d6b9 None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:16:00.926989 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46ec6f2-859a-4b04-88a9-90a08a8b72e8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8744c9de803048f583d0317d01f6a540', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:16:00.927535 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46ec6f2-859a-4b04-88a9-90a08a8b72e8 service nova] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:16:00.927814 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46ec6f2-859a-4b04-88a9-90a08a8b72e8 service nova] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:16:00.928084 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46ec6f2-859a-4b04-88a9-90a08a8b72e8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:16:00.929714 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-d46ec6f2-859a-4b04-88a9-90a08a8b72e8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:16:01.003701 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:16:01.016879 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46ec6f2-859a-4b04-88a9-90a08a8b72e8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:16:01.018967 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46ec6f2-859a-4b04-88a9-90a08a8b72e8 service nova] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:16:01.037690 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 724/1425] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:16:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:16:01.893048 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1c49ca-0e80-49f5-881d-9b6a76f9339d None None] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:16:01.893629 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1c49ca-0e80-49f5-881d-9b6a76f9339d None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:16:01.893629 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1c49ca-0e80-49f5-881d-9b6a76f9339d None None] PATH_INFO: `/` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:16:01.894962 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 702/1426] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 31 01:16:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:16:01.899971 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd917295-c257-4fd4-bd43-4080360337b4 None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:16:01.900405 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd917295-c257-4fd4-bd43-4080360337b4 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:16:01.900534 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd917295-c257-4fd4-bd43-4080360337b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:16:01.981909 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-bd917295-c257-4fd4-bd43-4080360337b4 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:16:01.983135 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-bd917295-c257-4fd4-bd43-4080360337b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:16:02.078498 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 725/1427] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 31 01:16:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:16:02.229950 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1c49ca-0e80-49f5-881d-9b6a76f9339d None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:16:02.231477 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-de1c49ca-0e80-49f5-881d-9b6a76f9339d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:16:02.314398 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:16:02.375391 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1c49ca-0e80-49f5-881d-9b6a76f9339d None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:16:02.485931 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee447f6e-1bcd-4001-9893-0979420a6b63 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8744c9de803048f583d0317d01f6a540', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:16:02.486426 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee447f6e-1bcd-4001-9893-0979420a6b63 service nova] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:16:02.486702 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee447f6e-1bcd-4001-9893-0979420a6b63 service nova] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:16:02.487008 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee447f6e-1bcd-4001-9893-0979420a6b63 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:16:02.488630 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-ee447f6e-1bcd-4001-9893-0979420a6b63 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:16:02.536105 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:16:02.575736 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee447f6e-1bcd-4001-9893-0979420a6b63 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:16:02.577391 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee447f6e-1bcd-4001-9893-0979420a6b63 service nova] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:16:02.595996 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 703/1428] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:16:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:16:03.449021 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79803a8-12de-4177-b1e7-8fe3fa2b2d71 None None] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:16:03.449599 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79803a8-12de-4177-b1e7-8fe3fa2b2d71 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:16:03.449599 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79803a8-12de-4177-b1e7-8fe3fa2b2d71 None None] PATH_INFO: `/` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:16:03.450946 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 726/1429] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 31 01:16:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:16:03.456344 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d44afc8-c9a9-4b48-9d5c-4f6bba825967 None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:16:03.456817 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d44afc8-c9a9-4b48-9d5c-4f6bba825967 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:16:03.456973 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d44afc8-c9a9-4b48-9d5c-4f6bba825967 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:16:03.537518 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-5d44afc8-c9a9-4b48-9d5c-4f6bba825967 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:16:03.538823 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-5d44afc8-c9a9-4b48-9d5c-4f6bba825967 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:16:03.638961 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 704/1430] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 31 01:16:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:16:03.789871 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79803a8-12de-4177-b1e7-8fe3fa2b2d71 None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:16:03.791320 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-d79803a8-12de-4177-b1e7-8fe3fa2b2d71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:16:03.839006 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:16:03.935356 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79803a8-12de-4177-b1e7-8fe3fa2b2d71 None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:16:04.047366 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd6bba9-43ba-41aa-9229-03710c8f5c41 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8744c9de803048f583d0317d01f6a540', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:16:04.047907 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd6bba9-43ba-41aa-9229-03710c8f5c41 service nova] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:16:04.048187 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd6bba9-43ba-41aa-9229-03710c8f5c41 service nova] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:16:04.048460 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd6bba9-43ba-41aa-9229-03710c8f5c41 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:16:04.050099 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-efd6bba9-43ba-41aa-9229-03710c8f5c41 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:16:04.056815 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:16:04.138885 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd6bba9-43ba-41aa-9229-03710c8f5c41 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:16:04.140496 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd6bba9-43ba-41aa-9229-03710c8f5c41 service nova] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:16:04.162244 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 727/1431] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:16:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:16:05.038719 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e221ed36-a04e-4658-b091-578c5c2cf252 None None] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:16:05.039387 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e221ed36-a04e-4658-b091-578c5c2cf252 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:16:05.039387 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e221ed36-a04e-4658-b091-578c5c2cf252 None None] PATH_INFO: `/` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:16:05.040629 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 705/1432] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 31 01:16:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:16:05.045665 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd2b4e0-5c73-4228-9b0e-7c399087ab0d None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:16:05.046092 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd2b4e0-5c73-4228-9b0e-7c399087ab0d None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:16:05.046256 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd2b4e0-5c73-4228-9b0e-7c399087ab0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:16:05.121893 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-8fd2b4e0-5c73-4228-9b0e-7c399087ab0d None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:16:05.123106 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-8fd2b4e0-5c73-4228-9b0e-7c399087ab0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:16:05.189277 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:16:05.218646 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 728/1433] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 31 01:16:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:16:05.372435 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e221ed36-a04e-4658-b091-578c5c2cf252 None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:16:05.373991 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-e221ed36-a04e-4658-b091-578c5c2cf252 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:16:05.391564 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:16:05.519538 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e221ed36-a04e-4658-b091-578c5c2cf252 None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:16:05.606834 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:16:05.632906 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c79e4a5-61fb-4b2d-93d7-69eaf125054d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8744c9de803048f583d0317d01f6a540', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:16:05.633529 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c79e4a5-61fb-4b2d-93d7-69eaf125054d service nova] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:16:05.634078 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c79e4a5-61fb-4b2d-93d7-69eaf125054d service nova] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:16:05.634537 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c79e4a5-61fb-4b2d-93d7-69eaf125054d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:16:05.636402 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-4c79e4a5-61fb-4b2d-93d7-69eaf125054d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:16:05.725068 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c79e4a5-61fb-4b2d-93d7-69eaf125054d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:16:05.726691 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c79e4a5-61fb-4b2d-93d7-69eaf125054d service nova] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:16:05.745118 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 706/1434] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:16:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:16:06.586158 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54955bd-1454-4b25-8ab4-163830dea464 None None] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:16:06.586690 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54955bd-1454-4b25-8ab4-163830dea464 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:16:06.586690 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54955bd-1454-4b25-8ab4-163830dea464 None None] PATH_INFO: `/` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:16:06.588061 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 729/1435] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 31 01:16:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:16:06.592879 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fcee0f-3429-4710-a469-b46434bd405c None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:16:06.593316 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fcee0f-3429-4710-a469-b46434bd405c None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:16:06.593428 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fcee0f-3429-4710-a469-b46434bd405c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:16:06.669600 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-11fcee0f-3429-4710-a469-b46434bd405c None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:16:06.670843 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-11fcee0f-3429-4710-a469-b46434bd405c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:16:06.712586 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:16:06.766546 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 707/1436] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 31 01:16:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:16:06.915824 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54955bd-1454-4b25-8ab4-163830dea464 None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:16:06.917380 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-d54955bd-1454-4b25-8ab4-163830dea464 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:16:07.061007 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54955bd-1454-4b25-8ab4-163830dea464 None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:16:07.114880 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:16:07.171855 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcf70f2-9b7e-420f-994c-00b9d5fa1dab service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8744c9de803048f583d0317d01f6a540', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:16:07.172459 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcf70f2-9b7e-420f-994c-00b9d5fa1dab service nova] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:16:07.172762 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcf70f2-9b7e-420f-994c-00b9d5fa1dab service nova] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:16:07.173065 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcf70f2-9b7e-420f-994c-00b9d5fa1dab service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:16:07.174652 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-4dcf70f2-9b7e-420f-994c-00b9d5fa1dab service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:16:07.260193 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcf70f2-9b7e-420f-994c-00b9d5fa1dab service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:16:07.261802 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcf70f2-9b7e-420f-994c-00b9d5fa1dab service nova] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:16:07.279829 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 730/1437] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:16:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:16:08.152502 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613239f7-321a-43ba-96db-fe5328b1c279 None None] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:16:08.153066 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613239f7-321a-43ba-96db-fe5328b1c279 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:16:08.153066 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613239f7-321a-43ba-96db-fe5328b1c279 None None] PATH_INFO: `/` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:16:08.154369 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 708/1438] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 31 01:16:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:16:08.159529 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5a9a7b-38a6-4393-98b3-4f4cee216388 None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:16:08.159974 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5a9a7b-38a6-4393-98b3-4f4cee216388 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:16:08.160087 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5a9a7b-38a6-4393-98b3-4f4cee216388 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:16:08.234475 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-9d5a9a7b-38a6-4393-98b3-4f4cee216388 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:16:08.235690 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-9d5a9a7b-38a6-4393-98b3-4f4cee216388 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:16:08.237714 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:16:08.331978 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 731/1439] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 31 01:16:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:16:08.485185 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613239f7-321a-43ba-96db-fe5328b1c279 None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:16:08.487260 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-613239f7-321a-43ba-96db-fe5328b1c279 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:16:08.630722 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613239f7-321a-43ba-96db-fe5328b1c279 None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:16:08.659485 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:16:08.746315 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cd0b86-134d-4701-b654-46d7b17a5fb3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8744c9de803048f583d0317d01f6a540', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:16:08.746836 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cd0b86-134d-4701-b654-46d7b17a5fb3 service nova] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:16:08.747111 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cd0b86-134d-4701-b654-46d7b17a5fb3 service nova] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:16:08.747380 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cd0b86-134d-4701-b654-46d7b17a5fb3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:16:08.748991 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-95cd0b86-134d-4701-b654-46d7b17a5fb3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:16:08.835537 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cd0b86-134d-4701-b654-46d7b17a5fb3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:16:08.837192 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cd0b86-134d-4701-b654-46d7b17a5fb3 service nova] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:16:08.855419 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 709/1440] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:16:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:16:10.652712 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b4e4e8-9836-4a22-8fc8-cf26614e2fc0 None None] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:16:10.653298 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b4e4e8-9836-4a22-8fc8-cf26614e2fc0 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:16:10.653298 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b4e4e8-9836-4a22-8fc8-cf26614e2fc0 None None] PATH_INFO: `/` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:16:10.654612 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 732/1441] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 01:16:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:16:10.660405 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1971f491-f89c-4c35-8cd6-13db5c2dcb99 None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:16:10.660854 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1971f491-f89c-4c35-8cd6-13db5c2dcb99 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:16:10.661004 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1971f491-f89c-4c35-8cd6-13db5c2dcb99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:16:10.701575 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:16:10.727100 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-1971f491-f89c-4c35-8cd6-13db5c2dcb99 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:16:10.728290 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-1971f491-f89c-4c35-8cd6-13db5c2dcb99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:16:10.822117 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 710/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 01:16:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:16:11.106978 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b4e4e8-9836-4a22-8fc8-cf26614e2fc0 None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:16:11.108647 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-90b4e4e8-9836-4a22-8fc8-cf26614e2fc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:16:11.211060 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b4e4e8-9836-4a22-8fc8-cf26614e2fc0 None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:16:11.242064 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:16:11.286035 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002f5047-978d-43f9-848c-c57516b236e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0817c3320d5d41d8bba0550c8ae8dd60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:16:11.286638 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002f5047-978d-43f9-848c-c57516b236e2 service neutron] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:16:11.286919 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002f5047-978d-43f9-848c-c57516b236e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:16:11.287192 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002f5047-978d-43f9-848c-c57516b236e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:16:11.288821 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-002f5047-978d-43f9-848c-c57516b236e2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:16:11.395540 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002f5047-978d-43f9-848c-c57516b236e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:16:11.396937 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002f5047-978d-43f9-848c-c57516b236e2 service neutron] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:16:11.415356 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 733/1443] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:16:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:16:13.449926 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5bcb6a-9699-41cf-a3b2-5074a1e619a3 None None] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:16:13.450494 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5bcb6a-9699-41cf-a3b2-5074a1e619a3 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:16:13.450494 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5bcb6a-9699-41cf-a3b2-5074a1e619a3 None None] PATH_INFO: `/` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:16:13.451855 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 711/1444] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 01:16:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:16:13.457328 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68ffe70-de96-47b2-a52c-cf11cbfda34b None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:16:13.457766 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68ffe70-de96-47b2-a52c-cf11cbfda34b None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:16:13.457870 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68ffe70-de96-47b2-a52c-cf11cbfda34b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:16:13.528887 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-d68ffe70-de96-47b2-a52c-cf11cbfda34b None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:16:13.530099 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-d68ffe70-de96-47b2-a52c-cf11cbfda34b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:16:13.543342 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:16:13.624591 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 734/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 01:16:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:16:13.908291 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5bcb6a-9699-41cf-a3b2-5074a1e619a3 None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:16:13.909858 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-5f5bcb6a-9699-41cf-a3b2-5074a1e619a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:16:14.011758 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5bcb6a-9699-41cf-a3b2-5074a1e619a3 None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:16:14.024286 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:16:14.085991 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bcabab-f349-420a-b931-5a31b3983c10 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0817c3320d5d41d8bba0550c8ae8dd60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:16:14.086598 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bcabab-f349-420a-b931-5a31b3983c10 service neutron] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:16:14.086878 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bcabab-f349-420a-b931-5a31b3983c10 service neutron] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:16:14.087153 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bcabab-f349-420a-b931-5a31b3983c10 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:16:14.088812 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-e7bcabab-f349-420a-b931-5a31b3983c10 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:16:14.198108 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bcabab-f349-420a-b931-5a31b3983c10 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:16:14.199557 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bcabab-f349-420a-b931-5a31b3983c10 service neutron] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:16:14.218377 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 712/1446] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:16:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:16:16.447259 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7b033d-f87d-40b0-99b4-6fc3dc13960b None None] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:16:16.447833 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7b033d-f87d-40b0-99b4-6fc3dc13960b None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:16:16.447833 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7b033d-f87d-40b0-99b4-6fc3dc13960b None None] PATH_INFO: `/` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:16:16.449235 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 735/1447] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 01:16:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:16:16.455250 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929d219c-d371-4786-b1f3-db1807af44b5 None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:16:16.455671 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929d219c-d371-4786-b1f3-db1807af44b5 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:16:16.455843 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929d219c-d371-4786-b1f3-db1807af44b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:16:16.503480 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:16:16.521387 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-929d219c-d371-4786-b1f3-db1807af44b5 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:16:16.524014 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-929d219c-d371-4786-b1f3-db1807af44b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:16:16.618105 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 713/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 01:16:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:16:16.898348 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7b033d-f87d-40b0-99b4-6fc3dc13960b None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:16:16.899929 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-6b7b033d-f87d-40b0-99b4-6fc3dc13960b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:16:17.001461 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7b033d-f87d-40b0-99b4-6fc3dc13960b None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:16:17.040669 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:16:17.074513 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3a3c84-0c89-4526-9177-52879f736ed3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0817c3320d5d41d8bba0550c8ae8dd60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:16:17.075122 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3a3c84-0c89-4526-9177-52879f736ed3 service neutron] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:16:17.075406 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3a3c84-0c89-4526-9177-52879f736ed3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:16:17.075673 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3a3c84-0c89-4526-9177-52879f736ed3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:16:17.077310 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-0b3a3c84-0c89-4526-9177-52879f736ed3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:16:17.184771 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3a3c84-0c89-4526-9177-52879f736ed3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:16:17.186198 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3a3c84-0c89-4526-9177-52879f736ed3 service neutron] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:16:17.203668 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 736/1449] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:16:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:16:19.240252 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a5d5fe-c601-4610-8760-35f93807f319 None None] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:16:19.240840 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a5d5fe-c601-4610-8760-35f93807f319 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:16:19.240840 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a5d5fe-c601-4610-8760-35f93807f319 None None] PATH_INFO: `/` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:16:19.242186 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 714/1450] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 01:16:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:16:19.247355 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e6ffea-0d3d-4b09-8209-fb527b5f2391 None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:16:19.247781 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e6ffea-0d3d-4b09-8209-fb527b5f2391 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:16:19.247903 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e6ffea-0d3d-4b09-8209-fb527b5f2391 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:16:19.312208 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-f3e6ffea-0d3d-4b09-8209-fb527b5f2391 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:16:19.313456 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-f3e6ffea-0d3d-4b09-8209-fb527b5f2391 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:16:19.333699 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:16:19.408611 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 737/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 01:16:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:16:19.697425 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a5d5fe-c601-4610-8760-35f93807f319 None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:16:19.698968 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-23a5d5fe-c601-4610-8760-35f93807f319 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:16:19.803870 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a5d5fe-c601-4610-8760-35f93807f319 None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:16:19.823168 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:16:19.876938 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ec7287-63b9-4376-83a3-454061ef5043 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0817c3320d5d41d8bba0550c8ae8dd60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:16:19.877585 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ec7287-63b9-4376-83a3-454061ef5043 service neutron] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:16:19.877870 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ec7287-63b9-4376-83a3-454061ef5043 service neutron] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:16:19.878146 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ec7287-63b9-4376-83a3-454061ef5043 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:16:19.879773 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-56ec7287-63b9-4376-83a3-454061ef5043 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:16:19.988981 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ec7287-63b9-4376-83a3-454061ef5043 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:16:19.990442 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ec7287-63b9-4376-83a3-454061ef5043 service neutron] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:16:20.008521 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 715/1452] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:16:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:16:21.954457 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fa1480-3075-4231-9cc3-67b14fe34e02 None None] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:16:21.955237 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fa1480-3075-4231-9cc3-67b14fe34e02 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:16:21.955350 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fa1480-3075-4231-9cc3-67b14fe34e02 None None] PATH_INFO: `/` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:16:21.956698 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 738/1453] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 01:16:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:16:21.962211 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548b4f1f-2b9b-4a35-9b45-229e1f9f227c None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:16:21.966947 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548b4f1f-2b9b-4a35-9b45-229e1f9f227c None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:16:21.966947 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548b4f1f-2b9b-4a35-9b45-229e1f9f227c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:16:22.027320 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:16:22.029985 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-548b4f1f-2b9b-4a35-9b45-229e1f9f227c None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:16:22.031179 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-548b4f1f-2b9b-4a35-9b45-229e1f9f227c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:16:22.124488 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 716/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 01:16:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:16:22.405729 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fa1480-3075-4231-9cc3-67b14fe34e02 None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:16:22.407777 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-f0fa1480-3075-4231-9cc3-67b14fe34e02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:16:22.507835 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fa1480-3075-4231-9cc3-67b14fe34e02 None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:16:22.555362 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:16:22.579198 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0fb669-185d-485a-b9ed-823710da6994 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0817c3320d5d41d8bba0550c8ae8dd60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:16:22.580056 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0fb669-185d-485a-b9ed-823710da6994 service neutron] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:16:22.580332 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0fb669-185d-485a-b9ed-823710da6994 service neutron] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:16:22.580600 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0fb669-185d-485a-b9ed-823710da6994 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:16:22.582190 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-6e0fb669-185d-485a-b9ed-823710da6994 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:16:22.688166 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0fb669-185d-485a-b9ed-823710da6994 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:16:22.693319 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0fb669-185d-485a-b9ed-823710da6994 service neutron] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:16:22.710184 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 739/1455] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:16:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:16:24.639390 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a839fd55-725b-4cbe-9faa-c89979b3cf07 None None] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:16:24.639934 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a839fd55-725b-4cbe-9faa-c89979b3cf07 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:16:24.639934 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a839fd55-725b-4cbe-9faa-c89979b3cf07 None None] PATH_INFO: `/` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:16:24.641324 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 717/1456] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 01:16:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:16:24.646508 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a731423-8152-4652-8465-db9ee3fbeeeb None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:16:24.646903 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a731423-8152-4652-8465-db9ee3fbeeeb None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:16:24.647101 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a731423-8152-4652-8465-db9ee3fbeeeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:16:24.713376 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-3a731423-8152-4652-8465-db9ee3fbeeeb None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:16:24.714583 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-3a731423-8152-4652-8465-db9ee3fbeeeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:16:24.743526 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:16:24.808418 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 740/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 01:16:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:16:25.091978 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a839fd55-725b-4cbe-9faa-c89979b3cf07 None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:16:25.093570 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-a839fd55-725b-4cbe-9faa-c89979b3cf07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:16:25.197395 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a839fd55-725b-4cbe-9faa-c89979b3cf07 None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:16:25.226035 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:16:25.270655 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f61f452-112f-4549-8e36-fb457dc7428e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0817c3320d5d41d8bba0550c8ae8dd60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:16:25.271259 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f61f452-112f-4549-8e36-fb457dc7428e service neutron] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:16:25.271538 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f61f452-112f-4549-8e36-fb457dc7428e service neutron] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:16:25.271809 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f61f452-112f-4549-8e36-fb457dc7428e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:16:25.273422 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-1f61f452-112f-4549-8e36-fb457dc7428e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:16:25.380000 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f61f452-112f-4549-8e36-fb457dc7428e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:16:25.381477 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f61f452-112f-4549-8e36-fb457dc7428e service neutron] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:16:25.399478 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 718/1458] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:16:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:16:27.326836 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e40cae-d384-42ba-a787-ebd9954ce1e0 None None] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:16:27.327418 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e40cae-d384-42ba-a787-ebd9954ce1e0 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:16:27.327418 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e40cae-d384-42ba-a787-ebd9954ce1e0 None None] PATH_INFO: `/` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:16:27.329443 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 741/1459] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 01:16:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:16:27.334823 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603bdbf3-68b0-4e71-99ba-c326ed191a53 None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:16:27.335231 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603bdbf3-68b0-4e71-99ba-c326ed191a53 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:16:27.335384 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603bdbf3-68b0-4e71-99ba-c326ed191a53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:16:27.400257 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-603bdbf3-68b0-4e71-99ba-c326ed191a53 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:16:27.401478 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-603bdbf3-68b0-4e71-99ba-c326ed191a53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:16:27.409721 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:16:27.497565 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 719/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 01:16:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:16:27.777392 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e40cae-d384-42ba-a787-ebd9954ce1e0 None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:16:27.778970 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-35e40cae-d384-42ba-a787-ebd9954ce1e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:16:27.880958 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e40cae-d384-42ba-a787-ebd9954ce1e0 None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:16:27.935942 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:16:27.955127 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afe09a2-92d7-4761-84a1-4023be3edbd6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0817c3320d5d41d8bba0550c8ae8dd60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:16:27.955759 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afe09a2-92d7-4761-84a1-4023be3edbd6 service neutron] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:16:27.956036 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afe09a2-92d7-4761-84a1-4023be3edbd6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:16:27.956308 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afe09a2-92d7-4761-84a1-4023be3edbd6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:16:27.957932 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-5afe09a2-92d7-4761-84a1-4023be3edbd6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:16:28.064572 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afe09a2-92d7-4761-84a1-4023be3edbd6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:16:28.066016 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afe09a2-92d7-4761-84a1-4023be3edbd6 service neutron] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:16:28.084437 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 742/1461] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:16:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:16:30.107750 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79452229-9ac6-4a05-b3d8-5e0d29510a2d None None] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:16:30.108309 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79452229-9ac6-4a05-b3d8-5e0d29510a2d None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:16:30.108309 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79452229-9ac6-4a05-b3d8-5e0d29510a2d None None] PATH_INFO: `/` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:16:30.109736 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 720/1462] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 01:16:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:16:30.114832 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5048821d-6948-4966-ad30-d44e32aaad0a None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:16:30.115260 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5048821d-6948-4966-ad30-d44e32aaad0a None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:16:30.115402 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5048821d-6948-4966-ad30-d44e32aaad0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:16:30.297647 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-5048821d-6948-4966-ad30-d44e32aaad0a None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:16:30.298884 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-5048821d-6948-4966-ad30-d44e32aaad0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:16:30.335879 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:16:30.392156 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 743/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 01:16:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:16:30.671827 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79452229-9ac6-4a05-b3d8-5e0d29510a2d None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:16:30.673407 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-79452229-9ac6-4a05-b3d8-5e0d29510a2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:16:30.774828 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79452229-9ac6-4a05-b3d8-5e0d29510a2d None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:16:30.811210 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:16:30.846706 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac02a0e-2ad2-49ce-bdb8-0b8ff037bd77 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0817c3320d5d41d8bba0550c8ae8dd60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:16:30.847308 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac02a0e-2ad2-49ce-bdb8-0b8ff037bd77 service neutron] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:16:30.847587 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac02a0e-2ad2-49ce-bdb8-0b8ff037bd77 service neutron] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:16:30.847878 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac02a0e-2ad2-49ce-bdb8-0b8ff037bd77 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:16:30.849484 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-bac02a0e-2ad2-49ce-bdb8-0b8ff037bd77 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:16:30.957776 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac02a0e-2ad2-49ce-bdb8-0b8ff037bd77 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:16:30.959154 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac02a0e-2ad2-49ce-bdb8-0b8ff037bd77 service neutron] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:16:30.976669 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 721/1464] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:16:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:16:32.897361 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04900ed-3b8c-408b-99b5-fbd05e21fe88 None None] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:16:32.897753 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04900ed-3b8c-408b-99b5-fbd05e21fe88 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:16:32.897753 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04900ed-3b8c-408b-99b5-fbd05e21fe88 None None] PATH_INFO: `/` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:16:32.899127 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 744/1465] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 01:16:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:16:32.904396 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8bf169-9f80-431d-864b-6179e80a2ab7 None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:16:32.904858 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8bf169-9f80-431d-864b-6179e80a2ab7 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:16:32.905072 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8bf169-9f80-431d-864b-6179e80a2ab7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:16:32.972173 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-2c8bf169-9f80-431d-864b-6179e80a2ab7 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:16:32.973488 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-2c8bf169-9f80-431d-864b-6179e80a2ab7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:16:32.991822 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:16:33.069254 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 722/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 01:16:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:16:33.349099 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04900ed-3b8c-408b-99b5-fbd05e21fe88 None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:16:33.350650 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-f04900ed-3b8c-408b-99b5-fbd05e21fe88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:16:33.454282 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04900ed-3b8c-408b-99b5-fbd05e21fe88 None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:16:33.521318 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:16:33.527744 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6783e89e-17b2-451a-8403-5be78076f266 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0817c3320d5d41d8bba0550c8ae8dd60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:16:33.530457 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6783e89e-17b2-451a-8403-5be78076f266 service neutron] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:16:33.530457 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6783e89e-17b2-451a-8403-5be78076f266 service neutron] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:16:33.530457 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6783e89e-17b2-451a-8403-5be78076f266 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:16:33.530457 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-6783e89e-17b2-451a-8403-5be78076f266 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:16:33.636214 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6783e89e-17b2-451a-8403-5be78076f266 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:16:33.637612 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6783e89e-17b2-451a-8403-5be78076f266 service neutron] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:16:33.655570 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 745/1467] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:16:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:16:35.583086 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3128917-9e2e-41fb-be06-9c7d03e3eddc None None] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:16:35.583706 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3128917-9e2e-41fb-be06-9c7d03e3eddc None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:16:35.583706 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3128917-9e2e-41fb-be06-9c7d03e3eddc None None] PATH_INFO: `/` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:16:35.585071 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 723/1468] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 01:16:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 01:16:35.590237 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacae340-5eca-40e8-b1b7-fac675f66bb1 None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:16:35.590679 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacae340-5eca-40e8-b1b7-fac675f66bb1 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:16:35.590789 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacae340-5eca-40e8-b1b7-fac675f66bb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:16:35.656953 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-bacae340-5eca-40e8-b1b7-fac675f66bb1 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:16:35.658202 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-bacae340-5eca-40e8-b1b7-fac675f66bb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:16:35.706114 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:16:35.753771 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 746/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 01:16:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:16:36.040332 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3128917-9e2e-41fb-be06-9c7d03e3eddc None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:16:36.041981 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-d3128917-9e2e-41fb-be06-9c7d03e3eddc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:16:36.144515 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3128917-9e2e-41fb-be06-9c7d03e3eddc None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:16:36.191025 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:16:36.217371 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412ea2c4-6d33-4533-b5ed-beffa5d39a44 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0817c3320d5d41d8bba0550c8ae8dd60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:16:36.217986 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412ea2c4-6d33-4533-b5ed-beffa5d39a44 service neutron] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:16:36.218279 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412ea2c4-6d33-4533-b5ed-beffa5d39a44 service neutron] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:16:36.218595 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412ea2c4-6d33-4533-b5ed-beffa5d39a44 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:16:36.220391 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-412ea2c4-6d33-4533-b5ed-beffa5d39a44 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:16:36.333790 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412ea2c4-6d33-4533-b5ed-beffa5d39a44 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:16:36.335282 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412ea2c4-6d33-4533-b5ed-beffa5d39a44 service neutron] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:16:36.354936 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 724/1470] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:16:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:16:45.751913 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=22 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:16:46.352641 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=66 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:16:55.785618 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0875cb37-8eaf-4ce6-9f78-d0ee9b57c5e5 None None] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:16:55.786164 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0875cb37-8eaf-4ce6-9f78-d0ee9b57c5e5 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:16:55.786164 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0875cb37-8eaf-4ce6-9f78-d0ee9b57c5e5 None None] PATH_INFO: `/` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:16:55.787556 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 747/1471] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:16:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:16:55.792578 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae20ae3e-855e-4609-a257-89fea7d0dbbc None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:16:55.793056 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae20ae3e-855e-4609-a257-89fea7d0dbbc None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:16:55.793190 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae20ae3e-855e-4609-a257-89fea7d0dbbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:16:55.857793 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-ae20ae3e-855e-4609-a257-89fea7d0dbbc None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:16:55.858984 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-ae20ae3e-855e-4609-a257-89fea7d0dbbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:16:55.955671 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 725/1472] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:16:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:16:56.253202 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a769b4-6357-4e99-afaf-12a4d4df218a None None] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:16:56.253490 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a769b4-6357-4e99-afaf-12a4d4df218a None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:16:56.253766 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a769b4-6357-4e99-afaf-12a4d4df218a None None] PATH_INFO: `/` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:16:56.255089 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 748/1473] 10.222.0.20 () {54 vars in 967 bytes} [Wed Aug 31 01:16:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:16:56.259885 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eeb209-dd59-4b04-886f-0a6647245c7d None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:16:56.260175 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eeb209-dd59-4b04-886f-0a6647245c7d None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:16:56.260453 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eeb209-dd59-4b04-886f-0a6647245c7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:16:56.335524 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-f1eeb209-dd59-4b04-886f-0a6647245c7d None None] MFA Rules not processed for user `d8590c288028418ebe7a4163fd8039f1`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:16:56.336736 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-f1eeb209-dd59-4b04-886f-0a6647245c7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:16:56.357026 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:16:56.412089 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 726/1474] 10.222.0.20 () {58 vars in 1064 bytes} [Wed Aug 31 01:16:56 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:16:56.419635 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a769b4-6357-4e99-afaf-12a4d4df218a None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:16:56.421203 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-40a769b4-6357-4e99-afaf-12a4d4df218a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:16:56.522294 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a769b4-6357-4e99-afaf-12a4d4df218a None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:16:56.592684 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a0ba67-af35-454a-a935-edf0892f2748 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8590c288028418ebe7a4163fd8039f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:16:56.593311 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a0ba67-af35-454a-a935-edf0892f2748 service glance] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:16:56.593593 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a0ba67-af35-454a-a935-edf0892f2748 service glance] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:16:56.593866 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a0ba67-af35-454a-a935-edf0892f2748 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:16:56.595950 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-49a0ba67-af35-454a-a935-edf0892f2748 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:16:56.648149 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:16:56.703799 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a0ba67-af35-454a-a935-edf0892f2748 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:16:56.705590 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a0ba67-af35-454a-a935-edf0892f2748 service glance] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:16:56.724765 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 749/1475] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:16:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:16:58.693292 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d59be9e-f104-4743-926b-c35e0cf341ee None None] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:16:58.693838 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d59be9e-f104-4743-926b-c35e0cf341ee None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:16:58.693960 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d59be9e-f104-4743-926b-c35e0cf341ee None None] PATH_INFO: `/` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:16:58.695332 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 727/1476] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:16:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:16:58.700436 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529d614e-cb63-4c92-9e71-84db9104bf73 None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:16:58.700912 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529d614e-cb63-4c92-9e71-84db9104bf73 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:16:58.701093 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529d614e-cb63-4c92-9e71-84db9104bf73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:16:58.767296 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-529d614e-cb63-4c92-9e71-84db9104bf73 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:16:58.768484 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-529d614e-cb63-4c92-9e71-84db9104bf73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:16:58.862328 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 750/1477] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:16:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:16:59.133572 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d59be9e-f104-4743-926b-c35e0cf341ee None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:16:59.135050 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-0d59be9e-f104-4743-926b-c35e0cf341ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:16:59.259619 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d59be9e-f104-4743-926b-c35e0cf341ee None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:16:59.307356 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:16:59.351521 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3d0c7e-643f-4fe1-8bdf-7c1a94e38745 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:16:59.352135 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3d0c7e-643f-4fe1-8bdf-7c1a94e38745 admin admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:16:59.352417 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3d0c7e-643f-4fe1-8bdf-7c1a94e38745 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:16:59.352692 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3d0c7e-643f-4fe1-8bdf-7c1a94e38745 admin admin] PATH_INFO: `/v3/projects` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:16:59.353367 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3d0c7e-643f-4fe1-8bdf-7c1a94e38745 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:16:59.355145 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: WARNING py.warnings [None req-ec3d0c7e-643f-4fe1-8bdf-7c1a94e38745 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 31 01:16:59.355145 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: warnings.warn(msg) Aug 31 01:16:59.355145 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: Aug 31 01:16:59.356306 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3d0c7e-643f-4fe1-8bdf-7c1a94e38745 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:16:59.375213 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 728/1478] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 01:16:59 2022] GET /identity/v3/projects => generated 2553 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:17:01.306012 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179128de-bcc2-4d39-a0f8-c8d9c218f660 None None] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:17:01.306537 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179128de-bcc2-4d39-a0f8-c8d9c218f660 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:17:01.306628 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179128de-bcc2-4d39-a0f8-c8d9c218f660 None None] PATH_INFO: `/` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:17:01.307939 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 751/1479] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:17:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:17:01.313411 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe71d76-d459-41bb-98ca-886cec1beb00 None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:17:01.313807 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe71d76-d459-41bb-98ca-886cec1beb00 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:17:01.313965 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe71d76-d459-41bb-98ca-886cec1beb00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:17:01.379790 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-efe71d76-d459-41bb-98ca-886cec1beb00 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:17:01.381038 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-efe71d76-d459-41bb-98ca-886cec1beb00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:17:01.478363 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 729/1480] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:17:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:17:01.773526 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179128de-bcc2-4d39-a0f8-c8d9c218f660 None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:17:01.775076 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-179128de-bcc2-4d39-a0f8-c8d9c218f660 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:17:01.785524 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:17:01.920637 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179128de-bcc2-4d39-a0f8-c8d9c218f660 None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:17:01.998393 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:17:02.031722 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcfb308-1ec9-4e70-acdb-43560c75ba7c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8744c9de803048f583d0317d01f6a540', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:17:02.032325 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcfb308-1ec9-4e70-acdb-43560c75ba7c service nova] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:17:02.032612 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcfb308-1ec9-4e70-acdb-43560c75ba7c service nova] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:17:02.032941 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcfb308-1ec9-4e70-acdb-43560c75ba7c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:17:02.034542 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-7dcfb308-1ec9-4e70-acdb-43560c75ba7c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:17:02.121993 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dcfb308-1ec9-4e70-acdb-43560c75ba7c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:17:02.123611 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dcfb308-1ec9-4e70-acdb-43560c75ba7c service nova] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:17:02.142323 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 752/1481] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:17:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:17:04.241414 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b32ed6c-212c-4888-94e6-591782c5294f None None] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:17:04.241965 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b32ed6c-212c-4888-94e6-591782c5294f None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:17:04.241965 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b32ed6c-212c-4888-94e6-591782c5294f None None] PATH_INFO: `/` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:17:04.243322 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 730/1482] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:17:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:17:04.248479 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10eafbe6-a1c2-4acc-a84c-315060bb45e4 None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:17:04.248891 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10eafbe6-a1c2-4acc-a84c-315060bb45e4 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:17:04.249057 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10eafbe6-a1c2-4acc-a84c-315060bb45e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:17:04.313311 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-10eafbe6-a1c2-4acc-a84c-315060bb45e4 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:17:04.314510 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-10eafbe6-a1c2-4acc-a84c-315060bb45e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:17:04.360950 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:17:04.409343 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 753/1483] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:17:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:17:04.695332 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b32ed6c-212c-4888-94e6-591782c5294f None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:17:04.696873 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-3b32ed6c-212c-4888-94e6-591782c5294f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:17:04.720288 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:17:04.799357 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b32ed6c-212c-4888-94e6-591782c5294f None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:17:04.875325 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c0c090-c1c4-476f-aad0-7c13b71ed3da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0817c3320d5d41d8bba0550c8ae8dd60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:17:04.875933 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c0c090-c1c4-476f-aad0-7c13b71ed3da service neutron] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:17:04.876213 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c0c090-c1c4-476f-aad0-7c13b71ed3da service neutron] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:17:04.876484 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c0c090-c1c4-476f-aad0-7c13b71ed3da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:17:04.878116 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-67c0c090-c1c4-476f-aad0-7c13b71ed3da service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:17:04.969653 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:17:04.998224 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c0c090-c1c4-476f-aad0-7c13b71ed3da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:17:04.999682 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c0c090-c1c4-476f-aad0-7c13b71ed3da service neutron] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:17:05.017865 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 731/1484] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:17:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:17:06.973185 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651f837f-a241-4199-9cbb-7ab162b8d524 None None] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:17:06.973734 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651f837f-a241-4199-9cbb-7ab162b8d524 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:17:06.973734 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651f837f-a241-4199-9cbb-7ab162b8d524 None None] PATH_INFO: `/` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:17:06.975090 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 754/1485] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:17:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:17:06.980262 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97819b95-82bb-4826-8985-54f04e68bebd None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:17:06.980709 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97819b95-82bb-4826-8985-54f04e68bebd None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:17:06.980984 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97819b95-82bb-4826-8985-54f04e68bebd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:17:07.046110 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-97819b95-82bb-4826-8985-54f04e68bebd None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:17:07.047269 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-97819b95-82bb-4826-8985-54f04e68bebd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:17:07.142851 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 732/1486] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:17:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:17:07.430965 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651f837f-a241-4199-9cbb-7ab162b8d524 None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:17:07.432517 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-651f837f-a241-4199-9cbb-7ab162b8d524 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:17:07.532469 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651f837f-a241-4199-9cbb-7ab162b8d524 None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:17:07.603143 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:17:07.604940 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101d2e3a-0539-4ef1-a896-aa2f28b0a6a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0817c3320d5d41d8bba0550c8ae8dd60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:17:07.605550 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101d2e3a-0539-4ef1-a896-aa2f28b0a6a6 service neutron] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:17:07.605869 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101d2e3a-0539-4ef1-a896-aa2f28b0a6a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:17:07.606143 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101d2e3a-0539-4ef1-a896-aa2f28b0a6a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:17:07.607733 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-101d2e3a-0539-4ef1-a896-aa2f28b0a6a6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:17:07.713735 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101d2e3a-0539-4ef1-a896-aa2f28b0a6a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:17:07.715094 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101d2e3a-0539-4ef1-a896-aa2f28b0a6a6 service neutron] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:17:07.732430 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 755/1487] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:17:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:17:09.738437 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc458e57-4c69-454e-b3ca-9c955abf02c6 None None] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:17:09.739003 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc458e57-4c69-454e-b3ca-9c955abf02c6 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:17:09.739003 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc458e57-4c69-454e-b3ca-9c955abf02c6 None None] PATH_INFO: `/` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:17:09.740345 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 733/1488] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:17:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:17:09.745783 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07efe050-8db4-4dbf-b496-ee334c943a8c None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:17:09.746162 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07efe050-8db4-4dbf-b496-ee334c943a8c None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:17:09.746339 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07efe050-8db4-4dbf-b496-ee334c943a8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:17:09.810963 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-07efe050-8db4-4dbf-b496-ee334c943a8c None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:17:09.812170 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-07efe050-8db4-4dbf-b496-ee334c943a8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:17:09.863764 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:17:09.908257 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 756/1489] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:17:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:17:10.199926 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc458e57-4c69-454e-b3ca-9c955abf02c6 None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:17:10.201771 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-dc458e57-4c69-454e-b3ca-9c955abf02c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:17:10.238153 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:17:10.306695 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc458e57-4c69-454e-b3ca-9c955abf02c6 None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:17:10.379020 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c73d550-d68c-477b-9632-cc60dbd7f67f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0817c3320d5d41d8bba0550c8ae8dd60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:17:10.379632 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c73d550-d68c-477b-9632-cc60dbd7f67f service neutron] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:17:10.379914 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c73d550-d68c-477b-9632-cc60dbd7f67f service neutron] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:17:10.380196 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c73d550-d68c-477b-9632-cc60dbd7f67f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:17:10.381850 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-0c73d550-d68c-477b-9632-cc60dbd7f67f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:17:10.460265 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:17:10.489345 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c73d550-d68c-477b-9632-cc60dbd7f67f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:17:10.490794 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c73d550-d68c-477b-9632-cc60dbd7f67f service neutron] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:17:10.508434 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 734/1490] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:17:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:17:12.905688 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6962ab9-289f-4d4e-b65e-4e3fcc61076e None None] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:17:12.906128 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6962ab9-289f-4d4e-b65e-4e3fcc61076e None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:17:12.906236 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6962ab9-289f-4d4e-b65e-4e3fcc61076e None None] PATH_INFO: `/` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:17:12.908171 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 757/1491] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 01:17:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:17:12.913463 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7055337b-a58d-46fd-8235-f99ada2a0778 None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:17:12.913900 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7055337b-a58d-46fd-8235-f99ada2a0778 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:17:12.914060 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7055337b-a58d-46fd-8235-f99ada2a0778 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:17:12.982154 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-7055337b-a58d-46fd-8235-f99ada2a0778 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:17:12.983383 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-7055337b-a58d-46fd-8235-f99ada2a0778 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:17:13.080440 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 735/1492] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 01:17:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:17:13.364703 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6962ab9-289f-4d4e-b65e-4e3fcc61076e None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:17:13.366285 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-a6962ab9-289f-4d4e-b65e-4e3fcc61076e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:17:13.469083 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6962ab9-289f-4d4e-b65e-4e3fcc61076e None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:17:13.539828 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee29546b-d994-4087-b0b2-5a7183ceb573 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0817c3320d5d41d8bba0550c8ae8dd60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:17:13.540426 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee29546b-d994-4087-b0b2-5a7183ceb573 service neutron] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:17:13.540706 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee29546b-d994-4087-b0b2-5a7183ceb573 service neutron] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:17:13.541009 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee29546b-d994-4087-b0b2-5a7183ceb573 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:17:13.542580 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-ee29546b-d994-4087-b0b2-5a7183ceb573 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:17:13.551858 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:17:13.649402 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee29546b-d994-4087-b0b2-5a7183ceb573 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:17:13.650916 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee29546b-d994-4087-b0b2-5a7183ceb573 service neutron] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:17:13.669364 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 758/1493] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:17:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:17:23.077890 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=82 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:17:23.667069 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=50 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:18:57.375858 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e757c1b2-3356-4fea-a253-0639bb3ad679 None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:18:57.376467 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e757c1b2-3356-4fea-a253-0639bb3ad679 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:18:57.376467 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e757c1b2-3356-4fea-a253-0639bb3ad679 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:18:57.451356 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-e757c1b2-3356-4fea-a253-0639bb3ad679 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:18:57.452554 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-e757c1b2-3356-4fea-a253-0639bb3ad679 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:18:57.668698 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 736/1494] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:18:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 294 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:18:57.677672 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfa7274-fc82-4b7d-a53b-3eb821f1fd66 None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:18:57.678168 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfa7274-fc82-4b7d-a53b-3eb821f1fd66 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:18:57.678272 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfa7274-fc82-4b7d-a53b-3eb821f1fd66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:18:57.751909 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-8cfa7274-fc82-4b7d-a53b-3eb821f1fd66 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:18:57.753149 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-8cfa7274-fc82-4b7d-a53b-3eb821f1fd66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:18:57.846856 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 759/1495] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:18:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:18:57.851378 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e757c1b2-3356-4fea-a253-0639bb3ad679 None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:18:57.852565 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-e757c1b2-3356-4fea-a253-0639bb3ad679 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:18:57.864318 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e757c1b2-3356-4fea-a253-0639bb3ad679 None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:18:57.925449 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:18:57.958353 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90db899-157f-4e3e-b99f-69b4bc3d669a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:18:57.959128 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90db899-157f-4e3e-b99f-69b4bc3d669a admin admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:18:57.959624 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90db899-157f-4e3e-b99f-69b4bc3d669a admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:18:57.960196 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90db899-157f-4e3e-b99f-69b4bc3d669a admin admin] PATH_INFO: `/v3/domains` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:18:57.960890 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90db899-157f-4e3e-b99f-69b4bc3d669a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:18:57.961198 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90db899-157f-4e3e-b99f-69b4bc3d669a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:18:57.962207 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: WARNING py.warnings [None req-f90db899-157f-4e3e-b99f-69b4bc3d669a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 31 01:18:57.962207 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: warnings.warn(msg) Aug 31 01:18:57.962207 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: Aug 31 01:18:57.963094 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90db899-157f-4e3e-b99f-69b4bc3d669a admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:18:57.975044 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 737/1496] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Aug 31 01:18:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:18:57.983345 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfa7274-fc82-4b7d-a53b-3eb821f1fd66 None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:18:57.992388 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfa7274-fc82-4b7d-a53b-3eb821f1fd66 None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:18:58.052908 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:18:58.086847 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632e4f14-9959-49dd-bffa-3221b28176e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:18:58.087448 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632e4f14-9959-49dd-bffa-3221b28176e4 admin admin] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:18:58.087727 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632e4f14-9959-49dd-bffa-3221b28176e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:18:58.087997 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632e4f14-9959-49dd-bffa-3221b28176e4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:18:58.088628 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632e4f14-9959-49dd-bffa-3221b28176e4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:18:58.090389 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: WARNING py.warnings [None req-632e4f14-9959-49dd-bffa-3221b28176e4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 31 01:18:58.090389 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: warnings.warn(msg) Aug 31 01:18:58.090389 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: Aug 31 01:18:58.091570 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632e4f14-9959-49dd-bffa-3221b28176e4 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:18:58.217891 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 760/1497] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 01:18:57 2022] POST /identity/v3/projects => generated 379 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:18:58.222720 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90db899-157f-4e3e-b99f-69b4bc3d669a admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:18:58.224307 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-f90db899-157f-4e3e-b99f-69b4bc3d669a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:18:58.256509 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90db899-157f-4e3e-b99f-69b4bc3d669a admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:18:58.352523 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6352c9ba-5794-48c6-87c8-e28eb55e2075 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:18:58.353177 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6352c9ba-5794-48c6-87c8-e28eb55e2075 admin admin] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:18:58.353462 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6352c9ba-5794-48c6-87c8-e28eb55e2075 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:18:58.353740 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6352c9ba-5794-48c6-87c8-e28eb55e2075 admin admin] PATH_INFO: `/v3/users` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:18:58.354376 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6352c9ba-5794-48c6-87c8-e28eb55e2075 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:18:58.355434 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: WARNING py.warnings [None req-6352c9ba-5794-48c6-87c8-e28eb55e2075 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 31 01:18:58.355434 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: warnings.warn(msg) Aug 31 01:18:58.355434 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: Aug 31 01:18:58.356647 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6352c9ba-5794-48c6-87c8-e28eb55e2075 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:18:58.468663 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 738/1498] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 01:18:58 2022] POST /identity/v3/users => generated 290 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:18:58.472990 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632e4f14-9959-49dd-bffa-3221b28176e4 admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:18:58.474522 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-632e4f14-9959-49dd-bffa-3221b28176e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:18:58.535749 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632e4f14-9959-49dd-bffa-3221b28176e4 admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:18:58.617612 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:18:58.628403 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec40ce5-ec48-472c-948c-fc96f60f2d63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:18:58.629052 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec40ce5-ec48-472c-948c-fc96f60f2d63 admin admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:18:58.629332 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec40ce5-ec48-472c-948c-fc96f60f2d63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:18:58.629607 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec40ce5-ec48-472c-948c-fc96f60f2d63 admin admin] PATH_INFO: `/v3/roles` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:18:58.630261 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec40ce5-ec48-472c-948c-fc96f60f2d63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:18:58.631255 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: WARNING py.warnings [None req-bec40ce5-ec48-472c-948c-fc96f60f2d63 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 31 01:18:58.631255 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: warnings.warn(msg) Aug 31 01:18:58.631255 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: Aug 31 01:18:58.632140 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec40ce5-ec48-472c-948c-fc96f60f2d63 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:18:58.645394 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 761/1499] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:18:58 2022] GET /identity/v3/roles => generated 1420 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:18:58.649152 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6352c9ba-5794-48c6-87c8-e28eb55e2075 admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:18:58.650887 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-6352c9ba-5794-48c6-87c8-e28eb55e2075 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:18:58.662009 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6352c9ba-5794-48c6-87c8-e28eb55e2075 admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:18:58.687105 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:18:58.756310 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3a2ae5-0450-4c3c-b2ba-a073b136925b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:18:58.756858 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3a2ae5-0450-4c3c-b2ba-a073b136925b admin admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:18:58.757140 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3a2ae5-0450-4c3c-b2ba-a073b136925b admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:18:58.757413 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3a2ae5-0450-4c3c-b2ba-a073b136925b admin admin] PATH_INFO: `/v3/roles` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:18:58.758050 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3a2ae5-0450-4c3c-b2ba-a073b136925b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:18:58.759060 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: WARNING py.warnings [None req-2f3a2ae5-0450-4c3c-b2ba-a073b136925b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 31 01:18:58.759060 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: warnings.warn(msg) Aug 31 01:18:58.759060 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: Aug 31 01:18:58.759949 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3a2ae5-0450-4c3c-b2ba-a073b136925b admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:18:58.772375 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 739/1500] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:18:58 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:18:58.776675 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec40ce5-ec48-472c-948c-fc96f60f2d63 admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:18:58.778256 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-bec40ce5-ec48-472c-948c-fc96f60f2d63 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:18:58.811262 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec40ce5-ec48-472c-948c-fc96f60f2d63 admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:18:58.904055 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a799f4-93c9-4233-a0d5-df53405b1c34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:18:58.904715 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a799f4-93c9-4233-a0d5-df53405b1c34 admin admin] REQUEST_METHOD: `PUT` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:18:58.905020 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a799f4-93c9-4233-a0d5-df53405b1c34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:18:58.905297 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a799f4-93c9-4233-a0d5-df53405b1c34 admin admin] PATH_INFO: `/v3/projects/dcfc38e9fa074270a3e26a8bcec7fd56/users/7ca5dbbc79944015a75b80700e06a5f5/roles/d0b9768f4d014766b9326387f2e11f34` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:18:58.927169 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a799f4-93c9-4233-a0d5-df53405b1c34 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dcfc38e9fa074270a3e26a8bcec7fd56, user_id=7ca5dbbc79944015a75b80700e06a5f5, role_id=d0b9768f4d014766b9326387f2e11f34)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:18:58.928184 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: WARNING py.warnings [None req-c8a799f4-93c9-4233-a0d5-df53405b1c34 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 31 01:18:58.928184 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: warnings.warn(msg) Aug 31 01:18:58.928184 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: Aug 31 01:18:58.930170 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a799f4-93c9-4233-a0d5-df53405b1c34 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:18:59.019492 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 762/1501] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 01:18:58 2022] PUT /identity/v3/projects/dcfc38e9fa074270a3e26a8bcec7fd56/users/7ca5dbbc79944015a75b80700e06a5f5/roles/d0b9768f4d014766b9326387f2e11f34 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:18:59.031713 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a75e1a9-a5a5-4b9a-b552-566858cb55b2 None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:18:59.032067 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a75e1a9-a5a5-4b9a-b552-566858cb55b2 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:18:59.032350 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a75e1a9-a5a5-4b9a-b552-566858cb55b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:18:59.090112 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-9a75e1a9-a5a5-4b9a-b552-566858cb55b2 None None] MFA Rules not processed for user `7ca5dbbc79944015a75b80700e06a5f5`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:18:59.091307 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-9a75e1a9-a5a5-4b9a-b552-566858cb55b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:18:59.167073 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 740/1502] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:18:59 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:18:59.180161 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651d742f-c3da-4141-bf6b-b9c30b852318 None None] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:18:59.180452 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651d742f-c3da-4141-bf6b-b9c30b852318 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:18:59.180749 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651d742f-c3da-4141-bf6b-b9c30b852318 None None] PATH_INFO: `/` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:18:59.182106 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 763/1503] 10.222.0.20 () {54 vars in 964 bytes} [Wed Aug 31 01:18:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:18:59.186982 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e49d20-0a1d-4b85-bb2a-3d887ee05646 None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:18:59.187268 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e49d20-0a1d-4b85-bb2a-3d887ee05646 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:18:59.187541 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e49d20-0a1d-4b85-bb2a-3d887ee05646 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:18:59.219057 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:18:59.262234 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-16e49d20-0a1d-4b85-bb2a-3d887ee05646 None None] MFA Rules not processed for user `a5c8fc69c518412ba1aad61a9f4f558c`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:18:59.263417 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-16e49d20-0a1d-4b85-bb2a-3d887ee05646 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:18:59.338377 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 741/1504] 10.222.0.20 () {58 vars in 1061 bytes} [Wed Aug 31 01:18:59 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:18:59.348406 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651d742f-c3da-4141-bf6b-b9c30b852318 None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:18:59.350030 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-651d742f-c3da-4141-bf6b-b9c30b852318 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:18:59.399105 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:18:59.453397 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651d742f-c3da-4141-bf6b-b9c30b852318 None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:18:59.525542 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90443d6-20cc-48c8-aca3-1646d41ba45b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5c8fc69c518412ba1aad61a9f4f558c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:18:59.526138 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90443d6-20cc-48c8-aca3-1646d41ba45b service cinder] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:18:59.526420 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90443d6-20cc-48c8-aca3-1646d41ba45b service cinder] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:18:59.526691 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90443d6-20cc-48c8-aca3-1646d41ba45b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:18:59.528255 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-e90443d6-20cc-48c8-aca3-1646d41ba45b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:18:59.625783 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90443d6-20cc-48c8-aca3-1646d41ba45b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:18:59.627154 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90443d6-20cc-48c8-aca3-1646d41ba45b service cinder] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:18:59.631264 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:18:59.645457 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 764/1505] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:18:59 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:18:59.689760 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec3ddcb-d285-4978-85bd-3c379ec27e16 None None] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:18:59.690763 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec3ddcb-d285-4978-85bd-3c379ec27e16 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:18:59.690911 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec3ddcb-d285-4978-85bd-3c379ec27e16 None None] PATH_INFO: `/` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:18:59.692172 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 742/1506] 10.222.0.20 () {52 vars in 830 bytes} [Wed Aug 31 01:18:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:18:59.695690 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90443d6-20cc-48c8-aca3-1646d41ba45b service cinder] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:18:59.696966 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-e90443d6-20cc-48c8-aca3-1646d41ba45b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:18:59.793358 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90443d6-20cc-48c8-aca3-1646d41ba45b service cinder] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:18:59.884353 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3421a80-c3c9-469b-a414-04868b96c25b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:18:59.884936 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3421a80-c3c9-469b-a414-04868b96c25b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:18:59.885215 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3421a80-c3c9-469b-a414-04868b96c25b admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:18:59.885516 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3421a80-c3c9-469b-a414-04868b96c25b admin admin] PATH_INFO: `/v3/users/7ca5dbbc79944015a75b80700e06a5f5` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:18:59.897898 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3421a80-c3c9-469b-a414-04868b96c25b admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ca5dbbc79944015a75b80700e06a5f5)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:18:59.898900 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: WARNING py.warnings [None req-d3421a80-c3c9-469b-a414-04868b96c25b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 31 01:18:59.898900 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: warnings.warn(msg) Aug 31 01:18:59.898900 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: Aug 31 01:18:59.900108 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3421a80-c3c9-469b-a414-04868b96c25b admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:18:59.910233 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:19:00.104678 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-d3421a80-c3c9-469b-a414-04868b96c25b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ca5dbbc79944015a75b80700e06a5f5'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:19:00.127853 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-d3421a80-c3c9-469b-a414-04868b96c25b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ca5dbbc79944015a75b80700e06a5f5'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:19:00.128311 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-d3421a80-c3c9-469b-a414-04868b96c25b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ca5dbbc79944015a75b80700e06a5f5'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:19:00.143661 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-d3421a80-c3c9-469b-a414-04868b96c25b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ca5dbbc79944015a75b80700e06a5f5'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:19:00.238805 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-d3421a80-c3c9-469b-a414-04868b96c25b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ca5dbbc79944015a75b80700e06a5f5'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:19:00.244456 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 765/1507] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 01:18:59 2022] DELETE /identity/v3/users/7ca5dbbc79944015a75b80700e06a5f5 => generated 0 bytes in 549 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:19:00.248872 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa442f0-3652-48e3-aeb8-99cbba0619be None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:19:00.249161 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa442f0-3652-48e3-aeb8-99cbba0619be None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:19:00.249439 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa442f0-3652-48e3-aeb8-99cbba0619be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:19:00.305496 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-faa442f0-3652-48e3-aeb8-99cbba0619be None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:19:00.306694 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-faa442f0-3652-48e3-aeb8-99cbba0619be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:19:00.380979 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:19:00.393972 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 743/1508] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:19:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:19:00.404288 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3421a80-c3c9-469b-a414-04868b96c25b admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:19:00.405594 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-d3421a80-c3c9-469b-a414-04868b96c25b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:19:00.504903 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3421a80-c3c9-469b-a414-04868b96c25b admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:19:00.558100 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:19:00.577146 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52838888-82db-4780-a719-4b412bc5e5fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0817c3320d5d41d8bba0550c8ae8dd60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:19:00.577756 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52838888-82db-4780-a719-4b412bc5e5fb service neutron] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:19:00.578038 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52838888-82db-4780-a719-4b412bc5e5fb service neutron] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:19:00.578312 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52838888-82db-4780-a719-4b412bc5e5fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:19:00.580257 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-52838888-82db-4780-a719-4b412bc5e5fb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:19:00.693023 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52838888-82db-4780-a719-4b412bc5e5fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:19:00.694463 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52838888-82db-4780-a719-4b412bc5e5fb service neutron] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:19:00.712807 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 766/1509] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:19:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:19:01.202125 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa442f0-3652-48e3-aeb8-99cbba0619be None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:19:01.203472 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-faa442f0-3652-48e3-aeb8-99cbba0619be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:19:01.218645 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa442f0-3652-48e3-aeb8-99cbba0619be None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:19:01.317295 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005669a4-bd41-4276-be26-fe60556dc5f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:19:01.317932 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005669a4-bd41-4276-be26-fe60556dc5f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:19:01.318214 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005669a4-bd41-4276-be26-fe60556dc5f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:19:01.318490 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005669a4-bd41-4276-be26-fe60556dc5f4 admin admin] PATH_INFO: `/v3/projects/dcfc38e9fa074270a3e26a8bcec7fd56` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:19:01.328256 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005669a4-bd41-4276-be26-fe60556dc5f4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dcfc38e9fa074270a3e26a8bcec7fd56)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:19:01.329346 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: WARNING py.warnings [None req-005669a4-bd41-4276-be26-fe60556dc5f4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 31 01:19:01.329346 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: warnings.warn(msg) Aug 31 01:19:01.329346 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: Aug 31 01:19:01.330800 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005669a4-bd41-4276-be26-fe60556dc5f4 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:19:01.483298 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-005669a4-bd41-4276-be26-fe60556dc5f4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dcfc38e9fa074270a3e26a8bcec7fd56', 'request_id': 'req-005669a4-bd41-4276-be26-fe60556dc5f4'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:19:01.490366 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-005669a4-bd41-4276-be26-fe60556dc5f4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dcfc38e9fa074270a3e26a8bcec7fd56', 'request_id': 'req-005669a4-bd41-4276-be26-fe60556dc5f4'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:19:01.503544 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-005669a4-bd41-4276-be26-fe60556dc5f4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dcfc38e9fa074270a3e26a8bcec7fd56', 'request_id': 'req-005669a4-bd41-4276-be26-fe60556dc5f4'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:19:01.592647 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-005669a4-bd41-4276-be26-fe60556dc5f4 admin admin] The token cache is being invalidate because project dcfc38e9fa074270a3e26a8bcec7fd56 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81749) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 01:19:01.592898 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-005669a4-bd41-4276-be26-fe60556dc5f4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:19:01.593419 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-005669a4-bd41-4276-be26-fe60556dc5f4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:19:01.594722 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 744/1510] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 01:19:01 2022] DELETE /identity/v3/projects/dcfc38e9fa074270a3e26a8bcec7fd56 => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:19:10.710575 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=62 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:19:11.590677 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:20:26.982209 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdfc4bb-e400-46e2-87e0-4a1b4fc9947c None None] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:20:26.982824 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdfc4bb-e400-46e2-87e0-4a1b4fc9947c None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:20:26.982824 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdfc4bb-e400-46e2-87e0-4a1b4fc9947c None None] PATH_INFO: `/` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:20:26.984202 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 767/1511] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 31 01:20:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 01:20:26.989063 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734e1803-9224-4df9-8afc-52f4c1a3286f None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:20:26.989481 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734e1803-9224-4df9-8afc-52f4c1a3286f None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:20:26.989678 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734e1803-9224-4df9-8afc-52f4c1a3286f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:20:27.070113 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-734e1803-9224-4df9-8afc-52f4c1a3286f None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:20:27.071327 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-734e1803-9224-4df9-8afc-52f4c1a3286f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:20:27.175920 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 745/1512] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 31 01:20:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:20:27.327765 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdfc4bb-e400-46e2-87e0-4a1b4fc9947c None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:20:27.335187 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-9cdfc4bb-e400-46e2-87e0-4a1b4fc9947c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:20:27.446663 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdfc4bb-e400-46e2-87e0-4a1b4fc9947c None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:20:27.452029 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36676b2b-54d8-48e7-b0bd-6c7aa1c28fb0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8744c9de803048f583d0317d01f6a540', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:20:27.452631 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36676b2b-54d8-48e7-b0bd-6c7aa1c28fb0 service nova] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:20:27.452954 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36676b2b-54d8-48e7-b0bd-6c7aa1c28fb0 service nova] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:20:27.453236 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36676b2b-54d8-48e7-b0bd-6c7aa1c28fb0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:20:27.461430 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36676b2b-54d8-48e7-b0bd-6c7aa1c28fb0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:20:27.463020 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36676b2b-54d8-48e7-b0bd-6c7aa1c28fb0 service nova] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:20:27.473081 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 768/1513] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:20:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:20:28.297211 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3897773a-e16e-48d0-b457-641a26f7ccc0 None None] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:20:28.297824 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3897773a-e16e-48d0-b457-641a26f7ccc0 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:20:28.297824 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3897773a-e16e-48d0-b457-641a26f7ccc0 None None] PATH_INFO: `/` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:20:28.299232 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 746/1514] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 31 01:20:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:20:28.304118 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d914960-9f4e-4bf4-a40e-44d1b54c567a None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:20:28.304521 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d914960-9f4e-4bf4-a40e-44d1b54c567a None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:20:28.304675 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d914960-9f4e-4bf4-a40e-44d1b54c567a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:20:28.324264 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-1d914960-9f4e-4bf4-a40e-44d1b54c567a None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:20:28.325526 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-1d914960-9f4e-4bf4-a40e-44d1b54c567a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:20:28.334255 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 769/1515] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 31 01:20:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:20:28.489684 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3897773a-e16e-48d0-b457-641a26f7ccc0 None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:20:28.504760 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3897773a-e16e-48d0-b457-641a26f7ccc0 None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:20:28.510210 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f64953-1a56-47e7-a89d-67667a4835e6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8744c9de803048f583d0317d01f6a540', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:20:28.510813 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f64953-1a56-47e7-a89d-67667a4835e6 service nova] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:20:28.511097 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f64953-1a56-47e7-a89d-67667a4835e6 service nova] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:20:28.511379 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f64953-1a56-47e7-a89d-67667a4835e6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:20:28.520284 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f64953-1a56-47e7-a89d-67667a4835e6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:20:28.521988 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f64953-1a56-47e7-a89d-67667a4835e6 service nova] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:20:28.531880 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 747/1516] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:20:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:20:29.778890 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d373147-229b-4b91-8835-1f2739e276fd None None] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:20:29.779442 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d373147-229b-4b91-8835-1f2739e276fd None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:20:29.779442 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d373147-229b-4b91-8835-1f2739e276fd None None] PATH_INFO: `/v3` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:20:29.780712 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 770/1517] 10.222.0.85 () {54 vars in 925 bytes} [Wed Aug 31 01:20:29 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:20:29.787365 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1d9bae-e377-44f6-98c9-43d435a4a693 None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:20:29.787764 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1d9bae-e377-44f6-98c9-43d435a4a693 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:20:29.787925 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1d9bae-e377-44f6-98c9-43d435a4a693 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:20:29.847931 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-3f1d9bae-e377-44f6-98c9-43d435a4a693 None None] MFA Rules not processed for user `dea0fe5b1d6b405cb47b95192580303b`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:20:29.849164 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-3f1d9bae-e377-44f6-98c9-43d435a4a693 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:20:29.865566 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:20:29.919597 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 748/1518] 10.222.0.85 () {58 vars in 1015 bytes} [Wed Aug 31 01:20:29 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 01:20:33.956650 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b075e9-80f3-4eba-a0fc-e440aed0c038 None None] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:20:33.957129 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b075e9-80f3-4eba-a0fc-e440aed0c038 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:20:33.957239 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b075e9-80f3-4eba-a0fc-e440aed0c038 None None] PATH_INFO: `/` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:20:33.958545 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 771/1519] 10.222.0.20 () {54 vars in 961 bytes} [Wed Aug 31 01:20:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:20:33.964400 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741048d6-8d23-424f-a621-82463be224e8 None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:20:33.964853 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741048d6-8d23-424f-a621-82463be224e8 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:20:33.965200 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741048d6-8d23-424f-a621-82463be224e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:20:33.984378 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-741048d6-8d23-424f-a621-82463be224e8 None None] MFA Rules not processed for user `dea0fe5b1d6b405cb47b95192580303b`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:20:33.985607 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-741048d6-8d23-424f-a621-82463be224e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:20:33.993925 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 749/1520] 10.222.0.20 () {58 vars in 1058 bytes} [Wed Aug 31 01:20:33 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:20:34.001209 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b075e9-80f3-4eba-a0fc-e440aed0c038 None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:20:34.015039 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b075e9-80f3-4eba-a0fc-e440aed0c038 None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:20:34.020012 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2548ff31-12f8-4a90-97c4-4c1ee1f66476 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dea0fe5b1d6b405cb47b95192580303b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:20:34.020603 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2548ff31-12f8-4a90-97c4-4c1ee1f66476 service placement] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:20:34.020930 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2548ff31-12f8-4a90-97c4-4c1ee1f66476 service placement] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:20:34.021209 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2548ff31-12f8-4a90-97c4-4c1ee1f66476 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:20:34.030117 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2548ff31-12f8-4a90-97c4-4c1ee1f66476 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:20:34.031711 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2548ff31-12f8-4a90-97c4-4c1ee1f66476 service placement] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:20:34.041334 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 772/1521] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:20:34 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 01:20:39.752850 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9c0284-75a0-41e9-aab3-cf10ab705b15 None None] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:20:39.753430 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9c0284-75a0-41e9-aab3-cf10ab705b15 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:20:39.753430 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9c0284-75a0-41e9-aab3-cf10ab705b15 None None] PATH_INFO: `/` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:20:39.754791 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 750/1522] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 31 01:20:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:20:39.760097 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0535e01a-9649-4d5b-aae8-e6f094e1f1d7 None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:20:39.760552 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0535e01a-9649-4d5b-aae8-e6f094e1f1d7 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:20:39.760648 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0535e01a-9649-4d5b-aae8-e6f094e1f1d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:20:39.781734 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-0535e01a-9649-4d5b-aae8-e6f094e1f1d7 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:20:39.782939 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-0535e01a-9649-4d5b-aae8-e6f094e1f1d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:20:39.790792 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 773/1523] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 31 01:20:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:20:39.942081 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9c0284-75a0-41e9-aab3-cf10ab705b15 None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:20:39.956973 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9c0284-75a0-41e9-aab3-cf10ab705b15 None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:20:39.962985 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e8bd19-4d45-46ad-90ca-452820ad8f8e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8744c9de803048f583d0317d01f6a540', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:20:39.963576 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e8bd19-4d45-46ad-90ca-452820ad8f8e service nova] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:20:39.963858 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e8bd19-4d45-46ad-90ca-452820ad8f8e service nova] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:20:39.964134 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e8bd19-4d45-46ad-90ca-452820ad8f8e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:20:39.972078 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e8bd19-4d45-46ad-90ca-452820ad8f8e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:20:39.973709 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e8bd19-4d45-46ad-90ca-452820ad8f8e service nova] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:20:39.983114 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 751/1524] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:20:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:20:40.801036 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e4add5-f212-4e73-b51c-ca5478f21e0c None None] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:20:40.801695 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e4add5-f212-4e73-b51c-ca5478f21e0c None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:20:40.801695 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e4add5-f212-4e73-b51c-ca5478f21e0c None None] PATH_INFO: `/` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:20:40.803021 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 774/1525] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 31 01:20:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:20:40.808373 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cbad03-c702-4101-8d16-5411b06ad6b0 None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:20:40.808809 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cbad03-c702-4101-8d16-5411b06ad6b0 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:20:40.809012 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cbad03-c702-4101-8d16-5411b06ad6b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:20:40.827928 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-87cbad03-c702-4101-8d16-5411b06ad6b0 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:20:40.829155 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-87cbad03-c702-4101-8d16-5411b06ad6b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:20:40.837735 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 752/1526] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 31 01:20:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:20:40.991413 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e4add5-f212-4e73-b51c-ca5478f21e0c None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:20:41.005536 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e4add5-f212-4e73-b51c-ca5478f21e0c None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:20:41.011089 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d632d92b-a8c4-4a67-9d94-6bc0dc4a51f2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8744c9de803048f583d0317d01f6a540', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:20:41.011683 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d632d92b-a8c4-4a67-9d94-6bc0dc4a51f2 service nova] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:20:41.011966 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d632d92b-a8c4-4a67-9d94-6bc0dc4a51f2 service nova] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:20:41.012239 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d632d92b-a8c4-4a67-9d94-6bc0dc4a51f2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:20:41.020121 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d632d92b-a8c4-4a67-9d94-6bc0dc4a51f2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:20:41.021726 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d632d92b-a8c4-4a67-9d94-6bc0dc4a51f2 service nova] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:20:41.031579 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 775/1527] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:20:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:20:44.910482 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c3c431-4874-4501-9a0a-361ae4ecbf17 None None] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:20:44.910986 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c3c431-4874-4501-9a0a-361ae4ecbf17 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:20:44.911086 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c3c431-4874-4501-9a0a-361ae4ecbf17 None None] PATH_INFO: `/` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:20:44.912396 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 753/1528] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 01:20:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:20:44.918553 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7fc32b-4b60-4fa1-9e8a-5aee418199fa None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:20:44.918990 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7fc32b-4b60-4fa1-9e8a-5aee418199fa None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:20:44.919086 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7fc32b-4b60-4fa1-9e8a-5aee418199fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:20:44.938570 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-0f7fc32b-4b60-4fa1-9e8a-5aee418199fa None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:20:44.939704 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-0f7fc32b-4b60-4fa1-9e8a-5aee418199fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:20:44.948076 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 776/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 01:20:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:20:45.246526 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c3c431-4874-4501-9a0a-361ae4ecbf17 None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:20:45.248312 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-20c3c431-4874-4501-9a0a-361ae4ecbf17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:20:45.326994 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c3c431-4874-4501-9a0a-361ae4ecbf17 None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:20:45.330807 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277a8527-8e0e-4c14-983d-de042e50f674 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0817c3320d5d41d8bba0550c8ae8dd60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:20:45.331413 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277a8527-8e0e-4c14-983d-de042e50f674 service neutron] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:20:45.331697 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277a8527-8e0e-4c14-983d-de042e50f674 service neutron] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:20:45.331979 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277a8527-8e0e-4c14-983d-de042e50f674 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:20:45.340763 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277a8527-8e0e-4c14-983d-de042e50f674 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:20:45.342446 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277a8527-8e0e-4c14-983d-de042e50f674 service neutron] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:20:45.351554 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 754/1530] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:20:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:20:47.220207 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab96d7a-011e-4693-a053-d7b436893007 None None] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:20:47.220776 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab96d7a-011e-4693-a053-d7b436893007 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:20:47.220776 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab96d7a-011e-4693-a053-d7b436893007 None None] PATH_INFO: `/` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:20:47.222106 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 777/1531] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 01:20:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:20:47.227819 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008f393a-12ae-4efb-a35c-fcd3509fc53a None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:20:47.228314 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008f393a-12ae-4efb-a35c-fcd3509fc53a None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:20:47.228480 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008f393a-12ae-4efb-a35c-fcd3509fc53a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:20:47.248130 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-008f393a-12ae-4efb-a35c-fcd3509fc53a None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:20:47.249347 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-008f393a-12ae-4efb-a35c-fcd3509fc53a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:20:47.256994 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 755/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 01:20:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:20:47.549021 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab96d7a-011e-4693-a053-d7b436893007 None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:20:47.560200 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab96d7a-011e-4693-a053-d7b436893007 None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:20:47.563675 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e23b86-24b5-4482-9c0e-17e9326fb161 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0817c3320d5d41d8bba0550c8ae8dd60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:20:47.564266 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e23b86-24b5-4482-9c0e-17e9326fb161 service neutron] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:20:47.564542 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e23b86-24b5-4482-9c0e-17e9326fb161 service neutron] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:20:47.564843 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e23b86-24b5-4482-9c0e-17e9326fb161 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:20:47.573356 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e23b86-24b5-4482-9c0e-17e9326fb161 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:20:47.574762 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e23b86-24b5-4482-9c0e-17e9326fb161 service neutron] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:20:47.584106 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 778/1533] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:20:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:20:50.940138 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4c77da-daf5-4ab7-841f-e75f190f2a37 None None] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:20:50.940642 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4c77da-daf5-4ab7-841f-e75f190f2a37 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:20:50.940792 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4c77da-daf5-4ab7-841f-e75f190f2a37 None None] PATH_INFO: `/` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:20:50.942045 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 756/1534] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 01:20:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:20:50.947602 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1673c9eb-3bc6-4846-9912-a39e33bf118c None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:20:50.948029 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1673c9eb-3bc6-4846-9912-a39e33bf118c None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:20:50.948163 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1673c9eb-3bc6-4846-9912-a39e33bf118c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:20:50.955988 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=81 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:20:50.968479 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-1673c9eb-3bc6-4846-9912-a39e33bf118c None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:20:50.970816 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-1673c9eb-3bc6-4846-9912-a39e33bf118c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:20:50.977688 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 779/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 01:20:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:20:51.524614 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4c77da-daf5-4ab7-841f-e75f190f2a37 None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:20:51.526295 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-6d4c77da-daf5-4ab7-841f-e75f190f2a37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:20:51.589125 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:20:51.602209 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4c77da-daf5-4ab7-841f-e75f190f2a37 None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:20:51.605895 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b37ca4c-5f38-4aa9-b236-6ed3ae9ac1d1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5c8fc69c518412ba1aad61a9f4f558c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:20:51.606500 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b37ca4c-5f38-4aa9-b236-6ed3ae9ac1d1 service cinder] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:20:51.606780 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b37ca4c-5f38-4aa9-b236-6ed3ae9ac1d1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:20:51.607054 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b37ca4c-5f38-4aa9-b236-6ed3ae9ac1d1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:20:51.615754 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b37ca4c-5f38-4aa9-b236-6ed3ae9ac1d1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:20:51.617223 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b37ca4c-5f38-4aa9-b236-6ed3ae9ac1d1 service cinder] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:20:51.627132 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 757/1536] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:20:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:20:53.533590 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c806512-94ae-42e2-9411-aa367e59e874 None None] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:20:53.534186 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c806512-94ae-42e2-9411-aa367e59e874 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:20:53.534186 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c806512-94ae-42e2-9411-aa367e59e874 None None] PATH_INFO: `/` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:20:53.535485 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 780/1537] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 01:20:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:20:53.541292 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc374dae-ef4a-4aee-8aa6-b12a6dcea2a0 None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:20:53.541725 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc374dae-ef4a-4aee-8aa6-b12a6dcea2a0 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:20:53.541861 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc374dae-ef4a-4aee-8aa6-b12a6dcea2a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:20:53.561888 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-fc374dae-ef4a-4aee-8aa6-b12a6dcea2a0 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:20:53.563087 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-fc374dae-ef4a-4aee-8aa6-b12a6dcea2a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:20:53.571070 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 758/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 01:20:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:20:54.122535 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c806512-94ae-42e2-9411-aa367e59e874 None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:20:54.124397 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-1c806512-94ae-42e2-9411-aa367e59e874 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:20:54.137590 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c806512-94ae-42e2-9411-aa367e59e874 None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:20:54.146108 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902e3054-dc63-4e26-afae-2fe4f786616c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5c8fc69c518412ba1aad61a9f4f558c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:20:54.146108 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902e3054-dc63-4e26-afae-2fe4f786616c service cinder] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:20:54.146108 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902e3054-dc63-4e26-afae-2fe4f786616c service cinder] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:20:54.146108 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902e3054-dc63-4e26-afae-2fe4f786616c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:20:54.151402 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902e3054-dc63-4e26-afae-2fe4f786616c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:20:54.152902 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902e3054-dc63-4e26-afae-2fe4f786616c service cinder] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:20:54.162254 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 781/1539] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:20:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:21:03.557023 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=11 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:21:04.158202 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=8 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:22:17.051657 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca16e4b-ed98-4e5d-a220-f2e9c5cb469b None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:17.053433 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca16e4b-ed98-4e5d-a220-f2e9c5cb469b None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:17.054539 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca16e4b-ed98-4e5d-a220-f2e9c5cb469b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:17.075777 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-cca16e4b-ed98-4e5d-a220-f2e9c5cb469b None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:22:17.077382 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-cca16e4b-ed98-4e5d-a220-f2e9c5cb469b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:22:17.090241 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c7b8f8-9991-4932-9142-87e860fda11e None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:17.090864 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 759/1540] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:22:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:22:17.091362 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c7b8f8-9991-4932-9142-87e860fda11e None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:17.091830 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c7b8f8-9991-4932-9142-87e860fda11e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:17.100409 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59be383-34db-4354-b053-31845a02b65a None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:17.101299 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59be383-34db-4354-b053-31845a02b65a None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:17.101578 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59be383-34db-4354-b053-31845a02b65a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:17.126836 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-c59be383-34db-4354-b053-31845a02b65a None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:22:17.128299 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-c59be383-34db-4354-b053-31845a02b65a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:22:17.129947 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-82c7b8f8-9991-4932-9142-87e860fda11e None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:22:17.135229 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-82c7b8f8-9991-4932-9142-87e860fda11e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:22:17.144567 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 760/1541] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:22:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 01:22:17.146241 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213bc2a4-3c86-42ab-a6e5-ac91ae6e7bd4 None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:17.147148 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 782/1542] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:22:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:22:17.148136 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213bc2a4-3c86-42ab-a6e5-ac91ae6e7bd4 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:17.148574 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213bc2a4-3c86-42ab-a6e5-ac91ae6e7bd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:17.155576 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b77bb4-02ea-446b-8f70-b9f6db9bcafe None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:17.156190 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b77bb4-02ea-446b-8f70-b9f6db9bcafe None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:17.156691 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b77bb4-02ea-446b-8f70-b9f6db9bcafe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:17.181230 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-213bc2a4-3c86-42ab-a6e5-ac91ae6e7bd4 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:22:17.182435 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-213bc2a4-3c86-42ab-a6e5-ac91ae6e7bd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:22:17.189889 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-41b77bb4-02ea-446b-8f70-b9f6db9bcafe None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:22:17.191406 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-41b77bb4-02ea-446b-8f70-b9f6db9bcafe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:22:17.198998 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 761/1543] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:22:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:22:17.199923 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213bc2a4-3c86-42ab-a6e5-ac91ae6e7bd4 None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:22:17.212101 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 783/1544] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:22:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 01:22:17.213738 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c6c0a4-41ab-4049-b741-ca743125a574 None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:17.214260 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c6c0a4-41ab-4049-b741-ca743125a574 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:17.214776 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c6c0a4-41ab-4049-b741-ca743125a574 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:17.216236 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213bc2a4-3c86-42ab-a6e5-ac91ae6e7bd4 None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:22:17.226192 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8f4059-2c25-49e4-ac66-3c67985b987e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:22:17.226819 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8f4059-2c25-49e4-ac66-3c67985b987e admin admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:17.227100 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8f4059-2c25-49e4-ac66-3c67985b987e admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:17.227373 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8f4059-2c25-49e4-ac66-3c67985b987e admin admin] PATH_INFO: `/v3/domains` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:17.228024 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8f4059-2c25-49e4-ac66-3c67985b987e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:22:17.228329 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8f4059-2c25-49e4-ac66-3c67985b987e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:22:17.232411 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8f4059-2c25-49e4-ac66-3c67985b987e admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:22:17.246171 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 762/1545] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Aug 31 01:22:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:22:17.252006 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-e9c6c0a4-41ab-4049-b741-ca743125a574 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:22:17.253535 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-e9c6c0a4-41ab-4049-b741-ca743125a574 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:22:17.253915 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8f4059-2c25-49e4-ac66-3c67985b987e admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:22:17.268548 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8f4059-2c25-49e4-ac66-3c67985b987e admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:22:17.271813 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 784/1546] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:22:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:22:17.273524 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c47204-edac-45a1-8b75-52d9968fbc7e None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:17.274049 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c47204-edac-45a1-8b75-52d9968fbc7e None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:17.274563 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c47204-edac-45a1-8b75-52d9968fbc7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:17.282380 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984d25ad-16b1-479c-8dfe-550cd7a7d828 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:22:17.283058 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984d25ad-16b1-479c-8dfe-550cd7a7d828 admin admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:17.283297 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984d25ad-16b1-479c-8dfe-550cd7a7d828 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:17.283554 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984d25ad-16b1-479c-8dfe-550cd7a7d828 admin admin] PATH_INFO: `/v3/domains` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:17.284197 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984d25ad-16b1-479c-8dfe-550cd7a7d828 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:22:17.284497 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984d25ad-16b1-479c-8dfe-550cd7a7d828 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:22:17.286131 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984d25ad-16b1-479c-8dfe-550cd7a7d828 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:22:17.299786 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 763/1547] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Aug 31 01:22:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:22:17.300629 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-26c47204-edac-45a1-8b75-52d9968fbc7e None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:22:17.301137 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace69309-7827-424e-b273-b88424be111c None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:17.301421 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace69309-7827-424e-b273-b88424be111c None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:17.301697 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace69309-7827-424e-b273-b88424be111c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:17.302944 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-26c47204-edac-45a1-8b75-52d9968fbc7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:22:17.315458 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 785/1548] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:22:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:22:17.316645 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c47204-edac-45a1-8b75-52d9968fbc7e None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:22:17.327231 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c47204-edac-45a1-8b75-52d9968fbc7e None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:22:17.327850 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-ace69309-7827-424e-b273-b88424be111c None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:22:17.329980 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-ace69309-7827-424e-b273-b88424be111c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:22:17.332395 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5544f0b9-530d-4723-8425-b73e02558326 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:22:17.333040 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5544f0b9-530d-4723-8425-b73e02558326 admin admin] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:17.333325 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5544f0b9-530d-4723-8425-b73e02558326 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:17.333601 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5544f0b9-530d-4723-8425-b73e02558326 admin admin] PATH_INFO: `/v3/projects` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:17.334234 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5544f0b9-530d-4723-8425-b73e02558326 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:22:17.336494 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5544f0b9-530d-4723-8425-b73e02558326 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:22:17.343029 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 764/1549] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:22:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:22:17.344170 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace69309-7827-424e-b273-b88424be111c None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:22:17.355665 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace69309-7827-424e-b273-b88424be111c None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:22:17.360456 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1e51e4-686c-4b30-9760-d91a2e419f2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:22:17.361100 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1e51e4-686c-4b30-9760-d91a2e419f2b admin admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:17.361391 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1e51e4-686c-4b30-9760-d91a2e419f2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:17.361667 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1e51e4-686c-4b30-9760-d91a2e419f2b admin admin] PATH_INFO: `/v3/domains` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:17.362316 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1e51e4-686c-4b30-9760-d91a2e419f2b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:22:17.362621 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1e51e4-686c-4b30-9760-d91a2e419f2b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:22:17.364219 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1e51e4-686c-4b30-9760-d91a2e419f2b admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:22:17.376558 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 765/1550] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Aug 31 01:22:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:22:17.377813 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46537ea5-fd80-4b29-b1ff-3715d89a8628 None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:17.378101 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46537ea5-fd80-4b29-b1ff-3715d89a8628 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:17.378378 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46537ea5-fd80-4b29-b1ff-3715d89a8628 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:17.396954 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-46537ea5-fd80-4b29-b1ff-3715d89a8628 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:22:17.398238 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-46537ea5-fd80-4b29-b1ff-3715d89a8628 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:22:17.406044 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 766/1551] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:22:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:22:17.406812 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46537ea5-fd80-4b29-b1ff-3715d89a8628 None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:22:17.416303 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46537ea5-fd80-4b29-b1ff-3715d89a8628 None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:22:17.420996 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4bd4ee-8001-4e48-bf85-997acfe5f643 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:22:17.426275 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4bd4ee-8001-4e48-bf85-997acfe5f643 admin admin] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:17.426275 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4bd4ee-8001-4e48-bf85-997acfe5f643 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:17.426275 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4bd4ee-8001-4e48-bf85-997acfe5f643 admin admin] PATH_INFO: `/v3/projects` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:17.426275 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4bd4ee-8001-4e48-bf85-997acfe5f643 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:22:17.426275 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: WARNING py.warnings [None req-3e4bd4ee-8001-4e48-bf85-997acfe5f643 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 31 01:22:17.426275 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: warnings.warn(msg) Aug 31 01:22:17.426275 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: Aug 31 01:22:17.426998 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4bd4ee-8001-4e48-bf85-997acfe5f643 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:22:17.452743 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 786/1552] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 01:22:17 2022] POST /identity/v3/projects => generated 365 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:22:17.459985 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5455b28-ff25-4e2b-8909-5c40c3042be7 None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:17.459985 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5455b28-ff25-4e2b-8909-5c40c3042be7 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:17.459985 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5455b28-ff25-4e2b-8909-5c40c3042be7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:17.473463 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-c5455b28-ff25-4e2b-8909-5c40c3042be7 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:22:17.474666 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-c5455b28-ff25-4e2b-8909-5c40c3042be7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:22:17.523722 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 767/1553] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 01:22:17 2022] POST /identity/v3/projects => generated 383 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:22:17.523722 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33603556-295d-4e3e-8fc4-9dbaa4c82d20 None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:17.523722 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33603556-295d-4e3e-8fc4-9dbaa4c82d20 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:17.523722 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33603556-295d-4e3e-8fc4-9dbaa4c82d20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:17.533270 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 787/1554] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:22:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:22:17.534514 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593e909b-3c2a-43d9-8015-e1e175e455a5 None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:17.534801 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593e909b-3c2a-43d9-8015-e1e175e455a5 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:17.535123 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593e909b-3c2a-43d9-8015-e1e175e455a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:17.547071 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-33603556-295d-4e3e-8fc4-9dbaa4c82d20 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:22:17.548415 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-33603556-295d-4e3e-8fc4-9dbaa4c82d20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:22:17.555271 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-593e909b-3c2a-43d9-8015-e1e175e455a5 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:22:17.556429 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-593e909b-3c2a-43d9-8015-e1e175e455a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:22:17.607940 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 768/1555] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:22:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:22:17.609384 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af13448-a986-4586-bc96-e0c6f0b07327 None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:17.609802 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af13448-a986-4586-bc96-e0c6f0b07327 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:17.610187 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af13448-a986-4586-bc96-e0c6f0b07327 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:17.615632 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 788/1556] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:22:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:22:17.617216 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593e909b-3c2a-43d9-8015-e1e175e455a5 None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:22:17.628765 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593e909b-3c2a-43d9-8015-e1e175e455a5 None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:22:17.630342 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-1af13448-a986-4586-bc96-e0c6f0b07327 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:22:17.632318 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-1af13448-a986-4586-bc96-e0c6f0b07327 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:22:17.634053 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a30ce8d-0082-424e-bfd4-ccd5f8b8cdd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:22:17.634676 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a30ce8d-0082-424e-bfd4-ccd5f8b8cdd0 admin admin] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:17.634947 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a30ce8d-0082-424e-bfd4-ccd5f8b8cdd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:17.635224 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a30ce8d-0082-424e-bfd4-ccd5f8b8cdd0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:17.635855 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a30ce8d-0082-424e-bfd4-ccd5f8b8cdd0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:22:17.637811 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a30ce8d-0082-424e-bfd4-ccd5f8b8cdd0 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:22:17.641270 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 769/1557] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:22:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:22:17.642434 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f88cab-57da-4775-8072-4343d8765ee8 None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:17.642645 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f88cab-57da-4775-8072-4343d8765ee8 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:17.643440 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f88cab-57da-4775-8072-4343d8765ee8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:17.661977 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-83f88cab-57da-4775-8072-4343d8765ee8 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:22:17.663106 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-83f88cab-57da-4775-8072-4343d8765ee8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:22:17.670903 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 770/1558] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:22:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:22:17.671650 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f88cab-57da-4775-8072-4343d8765ee8 None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:22:17.682139 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f88cab-57da-4775-8072-4343d8765ee8 None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:22:17.686700 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ff4ea6-2e41-4544-9f2e-78a149648894 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:22:17.687276 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ff4ea6-2e41-4544-9f2e-78a149648894 admin admin] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:17.687562 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ff4ea6-2e41-4544-9f2e-78a149648894 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:17.687840 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ff4ea6-2e41-4544-9f2e-78a149648894 admin admin] PATH_INFO: `/v3/users` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:17.688461 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ff4ea6-2e41-4544-9f2e-78a149648894 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:22:17.690429 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ff4ea6-2e41-4544-9f2e-78a149648894 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:22:17.727402 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 789/1559] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 01:22:17 2022] POST /identity/v3/projects => generated 379 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:22:17.727402 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a30ce8d-0082-424e-bfd4-ccd5f8b8cdd0 admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:22:17.749366 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:22:17.779112 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 771/1560] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 01:22:17 2022] POST /identity/v3/users => generated 283 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:22:17.779112 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce883cd-63cb-4791-b727-6387d998589a None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:17.779112 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce883cd-63cb-4791-b727-6387d998589a None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:17.779112 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce883cd-63cb-4791-b727-6387d998589a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:17.787482 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a30ce8d-0082-424e-bfd4-ccd5f8b8cdd0 admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:22:17.792479 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d538fa1-2a97-4981-b23e-baf05ed14e70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:22:17.793134 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d538fa1-2a97-4981-b23e-baf05ed14e70 admin admin] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:17.793413 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d538fa1-2a97-4981-b23e-baf05ed14e70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:17.793689 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d538fa1-2a97-4981-b23e-baf05ed14e70 admin admin] PATH_INFO: `/v3/users` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:17.794320 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d538fa1-2a97-4981-b23e-baf05ed14e70 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:22:17.795338 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: WARNING py.warnings [None req-2d538fa1-2a97-4981-b23e-baf05ed14e70 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 31 01:22:17.795338 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: warnings.warn(msg) Aug 31 01:22:17.795338 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: Aug 31 01:22:17.796550 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d538fa1-2a97-4981-b23e-baf05ed14e70 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:22:17.796797 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-5ce883cd-63cb-4791-b727-6387d998589a None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:22:17.798060 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-5ce883cd-63cb-4791-b727-6387d998589a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:22:17.807146 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 772/1561] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:22:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:22:17.807863 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce883cd-63cb-4791-b727-6387d998589a None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:22:17.816999 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce883cd-63cb-4791-b727-6387d998589a None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:22:17.822235 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bfec34-fe72-4c87-9a51-b3e586804356 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:22:17.822811 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bfec34-fe72-4c87-9a51-b3e586804356 admin admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:17.823094 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bfec34-fe72-4c87-9a51-b3e586804356 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:17.823370 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bfec34-fe72-4c87-9a51-b3e586804356 admin admin] PATH_INFO: `/v3/domains` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:17.823999 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bfec34-fe72-4c87-9a51-b3e586804356 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:22:17.824304 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bfec34-fe72-4c87-9a51-b3e586804356 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:22:17.825922 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bfec34-fe72-4c87-9a51-b3e586804356 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:22:17.837535 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 773/1562] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Aug 31 01:22:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:22:17.838638 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f739fb1e-86cc-4ecf-be51-0b6561727be2 None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:17.838925 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f739fb1e-86cc-4ecf-be51-0b6561727be2 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:17.839202 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f739fb1e-86cc-4ecf-be51-0b6561727be2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:17.857534 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-f739fb1e-86cc-4ecf-be51-0b6561727be2 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:22:17.858712 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-f739fb1e-86cc-4ecf-be51-0b6561727be2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:22:17.867092 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 774/1563] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:22:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:22:17.867844 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f739fb1e-86cc-4ecf-be51-0b6561727be2 None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:22:17.876691 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f739fb1e-86cc-4ecf-be51-0b6561727be2 None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:22:17.879447 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 790/1564] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 01:22:17 2022] POST /identity/v3/users => generated 292 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:22:17.880154 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d538fa1-2a97-4981-b23e-baf05ed14e70 admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:22:17.884380 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e947cd8-19ba-46c4-9e60-1029aa7e309b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:22:17.885076 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e947cd8-19ba-46c4-9e60-1029aa7e309b admin admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:17.885434 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e947cd8-19ba-46c4-9e60-1029aa7e309b admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:17.885784 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e947cd8-19ba-46c4-9e60-1029aa7e309b admin admin] PATH_INFO: `/v3/domains` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:17.886494 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e947cd8-19ba-46c4-9e60-1029aa7e309b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:22:17.886796 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e947cd8-19ba-46c4-9e60-1029aa7e309b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:22:17.888387 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e947cd8-19ba-46c4-9e60-1029aa7e309b admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:22:17.890215 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d538fa1-2a97-4981-b23e-baf05ed14e70 admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:22:17.890971 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:22:17.895351 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581f4ddd-a405-4659-8c73-42e93147c254 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:22:17.896191 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581f4ddd-a405-4659-8c73-42e93147c254 admin admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:17.896477 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581f4ddd-a405-4659-8c73-42e93147c254 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:17.896801 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581f4ddd-a405-4659-8c73-42e93147c254 admin admin] PATH_INFO: `/v3/domains` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:17.897953 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581f4ddd-a405-4659-8c73-42e93147c254 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:22:17.901839 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581f4ddd-a405-4659-8c73-42e93147c254 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:22:17.901839 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: WARNING py.warnings [None req-581f4ddd-a405-4659-8c73-42e93147c254 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 31 01:22:17.901839 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: warnings.warn(msg) Aug 31 01:22:17.901839 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: Aug 31 01:22:17.901839 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581f4ddd-a405-4659-8c73-42e93147c254 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:22:17.903297 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 775/1565] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Aug 31 01:22:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:22:17.904114 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e947cd8-19ba-46c4-9e60-1029aa7e309b admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:22:17.914350 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e947cd8-19ba-46c4-9e60-1029aa7e309b admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:22:17.920766 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 791/1566] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Aug 31 01:22:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:22:17.920766 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581f4ddd-a405-4659-8c73-42e93147c254 admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:22:17.925926 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f90ae5d-8dbb-43e3-a8ce-5c272b13ae32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:22:17.931169 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f90ae5d-8dbb-43e3-a8ce-5c272b13ae32 admin admin] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:17.931169 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f90ae5d-8dbb-43e3-a8ce-5c272b13ae32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:17.931169 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f90ae5d-8dbb-43e3-a8ce-5c272b13ae32 admin admin] PATH_INFO: `/v3/users` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:17.931489 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581f4ddd-a405-4659-8c73-42e93147c254 admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:22:17.931579 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f90ae5d-8dbb-43e3-a8ce-5c272b13ae32 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:22:17.931579 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f90ae5d-8dbb-43e3-a8ce-5c272b13ae32 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:22:17.932927 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4fb3d8-b494-412f-b02c-64135d03304a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:22:17.933521 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4fb3d8-b494-412f-b02c-64135d03304a admin admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:17.933805 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4fb3d8-b494-412f-b02c-64135d03304a admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:17.934081 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4fb3d8-b494-412f-b02c-64135d03304a admin admin] PATH_INFO: `/v3/roles` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:17.934717 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4fb3d8-b494-412f-b02c-64135d03304a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:22:17.936299 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4fb3d8-b494-412f-b02c-64135d03304a admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:22:17.947032 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 792/1567] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:22:17 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:22:17.947766 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4fb3d8-b494-412f-b02c-64135d03304a admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:22:17.957627 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4fb3d8-b494-412f-b02c-64135d03304a admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:22:17.962989 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b864a788-9d86-426c-b814-d1cff8e52f0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:22:17.963560 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b864a788-9d86-426c-b814-d1cff8e52f0b admin admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:17.963841 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b864a788-9d86-426c-b814-d1cff8e52f0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:17.964112 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b864a788-9d86-426c-b814-d1cff8e52f0b admin admin] PATH_INFO: `/v3/domains` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:17.964781 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b864a788-9d86-426c-b814-d1cff8e52f0b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:22:17.965082 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b864a788-9d86-426c-b814-d1cff8e52f0b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:22:17.966626 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b864a788-9d86-426c-b814-d1cff8e52f0b admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:22:17.978233 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 793/1568] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Aug 31 01:22:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:22:17.978980 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b864a788-9d86-426c-b814-d1cff8e52f0b admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:22:17.989579 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b864a788-9d86-426c-b814-d1cff8e52f0b admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:22:17.994236 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8078f72-810f-49b3-941f-e36adf55bb9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:22:17.994810 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8078f72-810f-49b3-941f-e36adf55bb9a admin admin] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:17.995093 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8078f72-810f-49b3-941f-e36adf55bb9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:17.995371 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8078f72-810f-49b3-941f-e36adf55bb9a admin admin] PATH_INFO: `/v3/projects` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:17.996048 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8078f72-810f-49b3-941f-e36adf55bb9a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:22:17.997960 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8078f72-810f-49b3-941f-e36adf55bb9a admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:22:18.003338 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 776/1569] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 01:22:17 2022] POST /identity/v3/users => generated 290 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:22:18.003338 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f90ae5d-8dbb-43e3-a8ce-5c272b13ae32 admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:22:18.016508 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f90ae5d-8dbb-43e3-a8ce-5c272b13ae32 admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:22:18.021345 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ac2737-cb7f-47cc-a3ef-065237a06527 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:22:18.021996 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ac2737-cb7f-47cc-a3ef-065237a06527 admin admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:18.022376 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ac2737-cb7f-47cc-a3ef-065237a06527 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:18.022656 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ac2737-cb7f-47cc-a3ef-065237a06527 admin admin] PATH_INFO: `/v3/domains` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:18.023328 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ac2737-cb7f-47cc-a3ef-065237a06527 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:22:18.023631 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ac2737-cb7f-47cc-a3ef-065237a06527 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:22:18.025400 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ac2737-cb7f-47cc-a3ef-065237a06527 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:22:18.036830 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 777/1570] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Aug 31 01:22:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:22:18.037670 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ac2737-cb7f-47cc-a3ef-065237a06527 admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:22:18.047548 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ac2737-cb7f-47cc-a3ef-065237a06527 admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:22:18.052842 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a2ee02-820b-40e6-86a9-a8b1787986c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:22:18.054076 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a2ee02-820b-40e6-86a9-a8b1787986c8 admin admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:18.054477 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a2ee02-820b-40e6-86a9-a8b1787986c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:18.054750 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a2ee02-820b-40e6-86a9-a8b1787986c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:18.055405 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a2ee02-820b-40e6-86a9-a8b1787986c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:22:18.057045 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a2ee02-820b-40e6-86a9-a8b1787986c8 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:22:18.066837 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 778/1571] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:22:18 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:22:18.067719 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a2ee02-820b-40e6-86a9-a8b1787986c8 admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:22:18.079590 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a2ee02-820b-40e6-86a9-a8b1787986c8 admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:22:18.087094 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 794/1572] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 01:22:17 2022] POST /identity/v3/projects => generated 385 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:22:18.087228 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd4105e-8b63-45cb-853a-6f272baeb928 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:22:18.087355 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8078f72-810f-49b3-941f-e36adf55bb9a admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:22:18.087467 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd4105e-8b63-45cb-853a-6f272baeb928 admin admin] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:18.087467 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd4105e-8b63-45cb-853a-6f272baeb928 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:18.087467 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd4105e-8b63-45cb-853a-6f272baeb928 admin admin] PATH_INFO: `/v3/projects` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:18.087467 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd4105e-8b63-45cb-853a-6f272baeb928 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:22:18.088600 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd4105e-8b63-45cb-853a-6f272baeb928 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:22:18.148180 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8078f72-810f-49b3-941f-e36adf55bb9a admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:22:18.152879 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b035a4-0464-4186-956c-a8d51f58cb49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:22:18.153472 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b035a4-0464-4186-956c-a8d51f58cb49 admin admin] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:18.153755 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b035a4-0464-4186-956c-a8d51f58cb49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:18.154034 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b035a4-0464-4186-956c-a8d51f58cb49 admin admin] PATH_INFO: `/v3/projects` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:18.154693 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b035a4-0464-4186-956c-a8d51f58cb49 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:22:18.156594 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b035a4-0464-4186-956c-a8d51f58cb49 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:22:18.184866 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 779/1573] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 01:22:18 2022] POST /identity/v3/projects => generated 389 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:22:18.184866 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd4105e-8b63-45cb-853a-6f272baeb928 admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:22:18.242618 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd4105e-8b63-45cb-853a-6f272baeb928 admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:22:18.247220 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c51e9a6-85be-4d36-a1d4-6766158acd5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:22:18.247821 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c51e9a6-85be-4d36-a1d4-6766158acd5a admin admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:18.248105 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c51e9a6-85be-4d36-a1d4-6766158acd5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:18.248382 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c51e9a6-85be-4d36-a1d4-6766158acd5a admin admin] PATH_INFO: `/v3/roles` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:18.249105 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c51e9a6-85be-4d36-a1d4-6766158acd5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:22:18.250717 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c51e9a6-85be-4d36-a1d4-6766158acd5a admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:22:18.260679 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 780/1574] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:22:18 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:22:18.261455 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c51e9a6-85be-4d36-a1d4-6766158acd5a admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:22:18.270843 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c51e9a6-85be-4d36-a1d4-6766158acd5a admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:22:18.274919 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 795/1575] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 01:22:18 2022] POST /identity/v3/projects => generated 393 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:22:18.275645 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b035a4-0464-4186-956c-a8d51f58cb49 admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:22:18.275781 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da0344d-71f3-4ba2-aea9-3381f287685c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:22:18.276388 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da0344d-71f3-4ba2-aea9-3381f287685c admin admin] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:18.276678 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da0344d-71f3-4ba2-aea9-3381f287685c admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:18.276970 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da0344d-71f3-4ba2-aea9-3381f287685c admin admin] PATH_INFO: `/v3/projects` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:18.277589 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da0344d-71f3-4ba2-aea9-3381f287685c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:22:18.279485 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da0344d-71f3-4ba2-aea9-3381f287685c admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:22:18.319378 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:22:18.340165 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b035a4-0464-4186-956c-a8d51f58cb49 admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:22:18.344811 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2ad526-6fdb-4a63-92b2-f91214de6108 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:22:18.345418 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2ad526-6fdb-4a63-92b2-f91214de6108 admin admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:18.345702 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2ad526-6fdb-4a63-92b2-f91214de6108 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:18.346007 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2ad526-6fdb-4a63-92b2-f91214de6108 admin admin] PATH_INFO: `/v3/roles` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:18.346645 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2ad526-6fdb-4a63-92b2-f91214de6108 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:22:18.348238 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2ad526-6fdb-4a63-92b2-f91214de6108 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:22:18.358522 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 796/1576] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:22:18 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:22:18.359314 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2ad526-6fdb-4a63-92b2-f91214de6108 admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:22:18.368750 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2ad526-6fdb-4a63-92b2-f91214de6108 admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:22:18.374205 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e4e509-c113-4e02-852a-848ada7a17cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:22:18.374523 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e4e509-c113-4e02-852a-848ada7a17cf admin admin] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:18.374807 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e4e509-c113-4e02-852a-848ada7a17cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:18.375084 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e4e509-c113-4e02-852a-848ada7a17cf admin admin] PATH_INFO: `/v3/projects` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:18.375707 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e4e509-c113-4e02-852a-848ada7a17cf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:22:18.377712 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e4e509-c113-4e02-852a-848ada7a17cf admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:22:18.457777 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 781/1577] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 01:22:18 2022] POST /identity/v3/projects => generated 369 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:22:18.458609 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da0344d-71f3-4ba2-aea9-3381f287685c admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:22:18.495420 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 797/1578] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 01:22:18 2022] POST /identity/v3/projects => generated 381 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:22:18.495420 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e4e509-c113-4e02-852a-848ada7a17cf admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:22:18.523968 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da0344d-71f3-4ba2-aea9-3381f287685c admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:22:18.532749 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:22:18.555157 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e4e509-c113-4e02-852a-848ada7a17cf admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:22:18.559819 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737de9dd-e24a-47b6-9a6b-6c2448f172fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:22:18.560427 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737de9dd-e24a-47b6-9a6b-6c2448f172fb admin admin] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:18.560709 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737de9dd-e24a-47b6-9a6b-6c2448f172fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:18.561016 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737de9dd-e24a-47b6-9a6b-6c2448f172fb admin admin] PATH_INFO: `/v3/users` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:18.561646 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737de9dd-e24a-47b6-9a6b-6c2448f172fb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:22:18.563556 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737de9dd-e24a-47b6-9a6b-6c2448f172fb admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:22:18.581189 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149e30f3-0cc7-4e35-ab99-8964dff70e83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:22:18.581796 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149e30f3-0cc7-4e35-ab99-8964dff70e83 admin admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:18.582166 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149e30f3-0cc7-4e35-ab99-8964dff70e83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:18.582348 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149e30f3-0cc7-4e35-ab99-8964dff70e83 admin admin] PATH_INFO: `/v3/roles` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:18.583005 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149e30f3-0cc7-4e35-ab99-8964dff70e83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:22:18.585145 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149e30f3-0cc7-4e35-ab99-8964dff70e83 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:22:18.594698 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 782/1579] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:22:18 2022] GET /identity/v3/roles => generated 1420 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:22:18.595447 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149e30f3-0cc7-4e35-ab99-8964dff70e83 admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:22:18.605097 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149e30f3-0cc7-4e35-ab99-8964dff70e83 admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:22:18.609633 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6dcc31-7978-4205-ac19-80f546b23742 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:22:18.610179 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6dcc31-7978-4205-ac19-80f546b23742 admin admin] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:18.610456 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6dcc31-7978-4205-ac19-80f546b23742 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:18.610727 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6dcc31-7978-4205-ac19-80f546b23742 admin admin] PATH_INFO: `/v3/users` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:18.611319 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6dcc31-7978-4205-ac19-80f546b23742 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:22:18.613202 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6dcc31-7978-4205-ac19-80f546b23742 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:22:18.654572 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 798/1580] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 01:22:18 2022] POST /identity/v3/users => generated 293 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:22:18.655377 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737de9dd-e24a-47b6-9a6b-6c2448f172fb admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:22:18.665135 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737de9dd-e24a-47b6-9a6b-6c2448f172fb admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:22:18.669771 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153a28e2-5599-4ac9-b752-33a1ed43b3af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:22:18.670305 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153a28e2-5599-4ac9-b752-33a1ed43b3af admin admin] REQUEST_METHOD: `PUT` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:18.670582 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153a28e2-5599-4ac9-b752-33a1ed43b3af admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:18.670854 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153a28e2-5599-4ac9-b752-33a1ed43b3af admin admin] PATH_INFO: `/v3/projects/5c58218b6d974ee6bd1a873fd2d5bb16/users/000ebe06ff37469a81d0af9fb3e1ba28/roles/d0b9768f4d014766b9326387f2e11f34` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:18.685928 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153a28e2-5599-4ac9-b752-33a1ed43b3af admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c58218b6d974ee6bd1a873fd2d5bb16, user_id=000ebe06ff37469a81d0af9fb3e1ba28, role_id=d0b9768f4d014766b9326387f2e11f34)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:22:18.688692 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153a28e2-5599-4ac9-b752-33a1ed43b3af admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:22:18.706246 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 783/1581] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 01:22:18 2022] POST /identity/v3/users => generated 295 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:22:18.707114 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6dcc31-7978-4205-ac19-80f546b23742 admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:22:18.722370 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6dcc31-7978-4205-ac19-80f546b23742 admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:22:18.726381 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbea802c-bf74-4554-97d6-d1c470f624e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:22:18.726981 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbea802c-bf74-4554-97d6-d1c470f624e8 admin admin] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:18.727265 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbea802c-bf74-4554-97d6-d1c470f624e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:18.727543 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbea802c-bf74-4554-97d6-d1c470f624e8 admin admin] PATH_INFO: `/v3/users` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:18.728285 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbea802c-bf74-4554-97d6-d1c470f624e8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:22:18.730289 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbea802c-bf74-4554-97d6-d1c470f624e8 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:22:18.794447 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 799/1582] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 01:22:18 2022] PUT /identity/v3/projects/5c58218b6d974ee6bd1a873fd2d5bb16/users/000ebe06ff37469a81d0af9fb3e1ba28/roles/d0b9768f4d014766b9326387f2e11f34 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:22:18.795744 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153a28e2-5599-4ac9-b752-33a1ed43b3af admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:22:18.817108 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 784/1583] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 01:22:18 2022] POST /identity/v3/users => generated 297 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:22:18.818018 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbea802c-bf74-4554-97d6-d1c470f624e8 admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:22:18.827316 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbea802c-bf74-4554-97d6-d1c470f624e8 admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:22:18.862726 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153a28e2-5599-4ac9-b752-33a1ed43b3af admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:22:18.868119 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65335fd3-0d8c-466b-87bb-51a283f2ee99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:22:18.868752 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65335fd3-0d8c-466b-87bb-51a283f2ee99 admin admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:18.869042 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65335fd3-0d8c-466b-87bb-51a283f2ee99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:18.869324 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65335fd3-0d8c-466b-87bb-51a283f2ee99 admin admin] PATH_INFO: `/v3/roles` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:18.869958 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65335fd3-0d8c-466b-87bb-51a283f2ee99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:22:18.871564 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65335fd3-0d8c-466b-87bb-51a283f2ee99 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:22:18.881212 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 800/1584] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:22:18 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:22:18.882017 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65335fd3-0d8c-466b-87bb-51a283f2ee99 admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:22:18.885832 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676782f0-5640-4cff-9ff6-f27734f1e671 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:22:18.886444 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676782f0-5640-4cff-9ff6-f27734f1e671 admin admin] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:18.886729 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676782f0-5640-4cff-9ff6-f27734f1e671 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:18.886999 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676782f0-5640-4cff-9ff6-f27734f1e671 admin admin] PATH_INFO: `/v3/users` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:18.887622 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676782f0-5640-4cff-9ff6-f27734f1e671 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:22:18.889561 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676782f0-5640-4cff-9ff6-f27734f1e671 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:22:18.891473 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65335fd3-0d8c-466b-87bb-51a283f2ee99 admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:22:18.899473 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398df838-6ab0-4c7b-be37-6ce84200a103 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:22:18.900500 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398df838-6ab0-4c7b-be37-6ce84200a103 admin admin] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:18.901167 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398df838-6ab0-4c7b-be37-6ce84200a103 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:18.901455 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398df838-6ab0-4c7b-be37-6ce84200a103 admin admin] PATH_INFO: `/v3/users` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:18.902221 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398df838-6ab0-4c7b-be37-6ce84200a103 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:22:18.904174 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398df838-6ab0-4c7b-be37-6ce84200a103 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:22:18.915696 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:22:18.964867 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 785/1585] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 01:22:18 2022] POST /identity/v3/users => generated 285 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:22:18.965726 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676782f0-5640-4cff-9ff6-f27734f1e671 admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:22:18.975085 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676782f0-5640-4cff-9ff6-f27734f1e671 admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:22:18.980094 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cad2d67-c27f-4986-bc52-3f168f9836e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:22:18.980749 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cad2d67-c27f-4986-bc52-3f168f9836e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:18.981087 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cad2d67-c27f-4986-bc52-3f168f9836e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:18.981374 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cad2d67-c27f-4986-bc52-3f168f9836e9 admin admin] PATH_INFO: `/v3/projects/314a7f3b9a1c48e98ff114f51e82ff01/users/8b73724e8e94402d8d18777f57a257e3/roles/d0b9768f4d014766b9326387f2e11f34` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:18.994930 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 801/1586] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 01:22:18 2022] POST /identity/v3/users => generated 291 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:22:18.996784 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398df838-6ab0-4c7b-be37-6ce84200a103 admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:22:18.998376 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cad2d67-c27f-4986-bc52-3f168f9836e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=314a7f3b9a1c48e98ff114f51e82ff01, user_id=8b73724e8e94402d8d18777f57a257e3, role_id=d0b9768f4d014766b9326387f2e11f34)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:22:18.999390 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: WARNING py.warnings [None req-4cad2d67-c27f-4986-bc52-3f168f9836e9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 31 01:22:18.999390 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: warnings.warn(msg) Aug 31 01:22:18.999390 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: Aug 31 01:22:19.001385 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cad2d67-c27f-4986-bc52-3f168f9836e9 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:22:19.005368 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398df838-6ab0-4c7b-be37-6ce84200a103 admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:22:19.010161 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019b8bbb-f4b4-4a1c-8420-c150eae63457 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:22:19.010752 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019b8bbb-f4b4-4a1c-8420-c150eae63457 admin admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:19.011053 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019b8bbb-f4b4-4a1c-8420-c150eae63457 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:19.011326 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019b8bbb-f4b4-4a1c-8420-c150eae63457 admin admin] PATH_INFO: `/v3/roles` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:19.011951 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019b8bbb-f4b4-4a1c-8420-c150eae63457 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:22:19.013585 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019b8bbb-f4b4-4a1c-8420-c150eae63457 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:22:19.024220 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 802/1587] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:22:18 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:22:19.025088 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019b8bbb-f4b4-4a1c-8420-c150eae63457 admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:22:19.034477 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019b8bbb-f4b4-4a1c-8420-c150eae63457 admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:22:19.039738 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4482fbd-6fca-493b-8c5b-d88549c1a8dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:22:19.040306 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4482fbd-6fca-493b-8c5b-d88549c1a8dc admin admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:19.040588 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4482fbd-6fca-493b-8c5b-d88549c1a8dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:19.040901 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4482fbd-6fca-493b-8c5b-d88549c1a8dc admin admin] PATH_INFO: `/v3/roles` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:19.041520 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4482fbd-6fca-493b-8c5b-d88549c1a8dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:22:19.043054 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4482fbd-6fca-493b-8c5b-d88549c1a8dc admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:22:19.052544 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 803/1588] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:22:19 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:22:19.054279 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475c5667-3600-4f22-bb66-d2f4019e9049 None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:19.054569 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475c5667-3600-4f22-bb66-d2f4019e9049 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:19.054872 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475c5667-3600-4f22-bb66-d2f4019e9049 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:19.083934 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-475c5667-3600-4f22-bb66-d2f4019e9049 None None] MFA Rules not processed for user `000ebe06ff37469a81d0af9fb3e1ba28`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:22:19.085720 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-475c5667-3600-4f22-bb66-d2f4019e9049 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:22:19.086131 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 786/1589] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 01:22:18 2022] PUT /identity/v3/projects/314a7f3b9a1c48e98ff114f51e82ff01/users/8b73724e8e94402d8d18777f57a257e3/roles/d0b9768f4d014766b9326387f2e11f34 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:22:19.088527 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cad2d67-c27f-4986-bc52-3f168f9836e9 admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:22:19.129094 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:22:19.148713 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cad2d67-c27f-4986-bc52-3f168f9836e9 admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:22:19.153307 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bef6928-46c9-4040-9e23-a8bf939e2f7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:22:19.153919 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bef6928-46c9-4040-9e23-a8bf939e2f7c admin admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:19.154198 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bef6928-46c9-4040-9e23-a8bf939e2f7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:19.154475 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bef6928-46c9-4040-9e23-a8bf939e2f7c admin admin] PATH_INFO: `/v3/roles` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:19.155349 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bef6928-46c9-4040-9e23-a8bf939e2f7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:22:19.156710 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bef6928-46c9-4040-9e23-a8bf939e2f7c admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:22:19.163477 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 804/1590] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:22:19 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:22:19.163477 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475c5667-3600-4f22-bb66-d2f4019e9049 None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:22:19.170911 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475c5667-3600-4f22-bb66-d2f4019e9049 None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:22:19.175813 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d596ee3-84c6-4633-abfd-b372de1b9651 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:22:19.180478 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d596ee3-84c6-4633-abfd-b372de1b9651 admin admin] REQUEST_METHOD: `PUT` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:19.180478 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d596ee3-84c6-4633-abfd-b372de1b9651 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:19.180685 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 787/1591] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:22:19 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:22:19.180825 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d596ee3-84c6-4633-abfd-b372de1b9651 admin admin] PATH_INFO: `/v3/projects/3abb95e8d0a84b76a5d141815451ceae/users/b7d058c2b40343a6a3e2e46f6b2bab0e/roles/d0b9768f4d014766b9326387f2e11f34` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:19.180909 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bef6928-46c9-4040-9e23-a8bf939e2f7c admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:22:19.187595 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bef6928-46c9-4040-9e23-a8bf939e2f7c admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:22:19.191949 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d596ee3-84c6-4633-abfd-b372de1b9651 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3abb95e8d0a84b76a5d141815451ceae, user_id=b7d058c2b40343a6a3e2e46f6b2bab0e, role_id=d0b9768f4d014766b9326387f2e11f34)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:22:19.192605 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b259148-2076-4281-8407-87335e01f1ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:22:19.193235 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b259148-2076-4281-8407-87335e01f1ec admin admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:19.193518 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b259148-2076-4281-8407-87335e01f1ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:19.193806 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b259148-2076-4281-8407-87335e01f1ec admin admin] PATH_INFO: `/v3/roles` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:19.194451 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b259148-2076-4281-8407-87335e01f1ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:22:19.194637 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d596ee3-84c6-4633-abfd-b372de1b9651 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:22:19.196545 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b259148-2076-4281-8407-87335e01f1ec admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:22:19.207261 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 788/1592] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:22:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:22:19.208115 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b259148-2076-4281-8407-87335e01f1ec admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:22:19.218105 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b259148-2076-4281-8407-87335e01f1ec admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:22:19.222667 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde2a903-07e2-4683-ad84-43409989339a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:22:19.223225 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde2a903-07e2-4683-ad84-43409989339a admin admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:19.223509 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde2a903-07e2-4683-ad84-43409989339a admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:19.223787 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde2a903-07e2-4683-ad84-43409989339a admin admin] PATH_INFO: `/v3/roles` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:19.224397 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde2a903-07e2-4683-ad84-43409989339a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:22:19.226251 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde2a903-07e2-4683-ad84-43409989339a admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:22:19.236182 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 789/1593] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:22:19 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:22:19.237029 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde2a903-07e2-4683-ad84-43409989339a admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:22:19.246773 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde2a903-07e2-4683-ad84-43409989339a admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:22:19.251357 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a90d3c8-5ac7-476f-bee6-847879e5f954 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:22:19.251900 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a90d3c8-5ac7-476f-bee6-847879e5f954 admin admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:19.252183 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a90d3c8-5ac7-476f-bee6-847879e5f954 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:19.252461 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a90d3c8-5ac7-476f-bee6-847879e5f954 admin admin] PATH_INFO: `/v3/roles` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:19.253097 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a90d3c8-5ac7-476f-bee6-847879e5f954 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:22:19.254627 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a90d3c8-5ac7-476f-bee6-847879e5f954 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:22:19.262640 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 805/1594] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 01:22:19 2022] PUT /identity/v3/projects/3abb95e8d0a84b76a5d141815451ceae/users/b7d058c2b40343a6a3e2e46f6b2bab0e/roles/d0b9768f4d014766b9326387f2e11f34 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:22:19.263578 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d596ee3-84c6-4633-abfd-b372de1b9651 admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:22:19.265232 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 790/1595] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:22:19 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:22:19.266541 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd3499c-69e4-4189-9885-5a1661b91885 None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:19.266910 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd3499c-69e4-4189-9885-5a1661b91885 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:19.267189 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd3499c-69e4-4189-9885-5a1661b91885 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:19.295516 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-3fd3499c-69e4-4189-9885-5a1661b91885 None None] MFA Rules not processed for user `8b73724e8e94402d8d18777f57a257e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:22:19.296741 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-3fd3499c-69e4-4189-9885-5a1661b91885 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:22:19.327249 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d596ee3-84c6-4633-abfd-b372de1b9651 admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:22:19.332142 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e61cc0-4dc7-4e05-94e7-994c75b33f6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:22:19.332774 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e61cc0-4dc7-4e05-94e7-994c75b33f6b admin admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:19.333061 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e61cc0-4dc7-4e05-94e7-994c75b33f6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:19.333332 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e61cc0-4dc7-4e05-94e7-994c75b33f6b admin admin] PATH_INFO: `/v3/roles` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:19.333974 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e61cc0-4dc7-4e05-94e7-994c75b33f6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:22:19.335612 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e61cc0-4dc7-4e05-94e7-994c75b33f6b admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:22:19.345566 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 806/1596] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:22:19 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:22:19.346491 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e61cc0-4dc7-4e05-94e7-994c75b33f6b admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:22:19.366461 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 791/1597] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:22:19 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:22:19.369601 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd3499c-69e4-4189-9885-5a1661b91885 None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:22:19.381508 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd3499c-69e4-4189-9885-5a1661b91885 None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:22:19.386719 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45219b8-b469-46a7-8b8f-173e61a54313 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:22:19.387326 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45219b8-b469-46a7-8b8f-173e61a54313 admin admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:19.387610 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45219b8-b469-46a7-8b8f-173e61a54313 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:19.388091 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45219b8-b469-46a7-8b8f-173e61a54313 admin admin] PATH_INFO: `/v3/roles` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:19.389231 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45219b8-b469-46a7-8b8f-173e61a54313 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:22:19.390276 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:22:19.390465 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45219b8-b469-46a7-8b8f-173e61a54313 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:22:19.405534 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 792/1598] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:22:19 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:22:19.406475 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45219b8-b469-46a7-8b8f-173e61a54313 admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:22:19.414193 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e61cc0-4dc7-4e05-94e7-994c75b33f6b admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:22:19.416617 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45219b8-b469-46a7-8b8f-173e61a54313 admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:22:19.417939 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a918f02-356b-4f12-ba62-5994a416222b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5c8fc69c518412ba1aad61a9f4f558c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:22:19.418663 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a918f02-356b-4f12-ba62-5994a416222b service cinder] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:19.419012 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a918f02-356b-4f12-ba62-5994a416222b service cinder] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:19.419365 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a918f02-356b-4f12-ba62-5994a416222b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:19.421207 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdfb7f1-3cd2-4e33-b313-f4bcc1071d1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:22:19.421790 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdfb7f1-3cd2-4e33-b313-f4bcc1071d1b admin admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:19.422068 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdfb7f1-3cd2-4e33-b313-f4bcc1071d1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:19.422339 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdfb7f1-3cd2-4e33-b313-f4bcc1071d1b admin admin] PATH_INFO: `/v3/roles` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:19.422970 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdfb7f1-3cd2-4e33-b313-f4bcc1071d1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:22:19.424581 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdfb7f1-3cd2-4e33-b313-f4bcc1071d1b admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:22:19.434920 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 793/1599] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:22:19 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:22:19.435707 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdfb7f1-3cd2-4e33-b313-f4bcc1071d1b admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:22:19.445047 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdfb7f1-3cd2-4e33-b313-f4bcc1071d1b admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:22:19.449841 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb7737f-a9a7-45b9-905f-da265d3e0bcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:22:19.450411 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb7737f-a9a7-45b9-905f-da265d3e0bcb admin admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:19.450688 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb7737f-a9a7-45b9-905f-da265d3e0bcb admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:19.450960 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb7737f-a9a7-45b9-905f-da265d3e0bcb admin admin] PATH_INFO: `/v3/roles` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:19.451577 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb7737f-a9a7-45b9-905f-da265d3e0bcb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:22:19.453194 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb7737f-a9a7-45b9-905f-da265d3e0bcb admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:22:19.463755 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 794/1600] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:22:19 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:22:19.464959 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0de0466-0ba0-487d-aacb-0956db9c1645 None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:19.465280 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0de0466-0ba0-487d-aacb-0956db9c1645 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:19.465569 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0de0466-0ba0-487d-aacb-0956db9c1645 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:19.482353 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a918f02-356b-4f12-ba62-5994a416222b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:22:19.483828 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a918f02-356b-4f12-ba62-5994a416222b service cinder] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:22:19.484015 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-b0de0466-0ba0-487d-aacb-0956db9c1645 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:22:19.485224 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-b0de0466-0ba0-487d-aacb-0956db9c1645 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:22:19.493307 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 795/1601] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:22:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:22:19.501277 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 807/1602] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:22:19 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:22:19.501633 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0de0466-0ba0-487d-aacb-0956db9c1645 None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:22:19.501736 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a918f02-356b-4f12-ba62-5994a416222b service cinder] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:22:19.505345 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a918f02-356b-4f12-ba62-5994a416222b service cinder] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:22:19.507069 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0de0466-0ba0-487d-aacb-0956db9c1645 None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:22:19.510549 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3aa6166-d6c1-4987-ac6e-d3265f2af21a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:22:19.511212 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3aa6166-d6c1-4987-ac6e-d3265f2af21a admin admin] REQUEST_METHOD: `PUT` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:19.511497 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3aa6166-d6c1-4987-ac6e-d3265f2af21a admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:19.511779 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3aa6166-d6c1-4987-ac6e-d3265f2af21a admin admin] PATH_INFO: `/v3/projects/53d2295ada114ea3b6b5001c4c197f35/users/efaf57b6482149ca86a9bba26d7654f2/roles/d0b9768f4d014766b9326387f2e11f34` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:19.514276 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82aef70-15fe-4571-9a78-0a7641f0deb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:22:19.515915 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82aef70-15fe-4571-9a78-0a7641f0deb4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:19.516214 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82aef70-15fe-4571-9a78-0a7641f0deb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:19.516500 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82aef70-15fe-4571-9a78-0a7641f0deb4 admin admin] PATH_INFO: `/v3/projects/164da7d4776d4e9f8cf6497aa784d312/users/52ed99737fd240789bb682fd9bf11be3/roles/d0b9768f4d014766b9326387f2e11f34` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:19.527681 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3aa6166-d6c1-4987-ac6e-d3265f2af21a admin admin] RBAC: Authorizing `identity:create_grant(project_id=53d2295ada114ea3b6b5001c4c197f35, user_id=efaf57b6482149ca86a9bba26d7654f2, role_id=d0b9768f4d014766b9326387f2e11f34)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:22:19.530753 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3aa6166-d6c1-4987-ac6e-d3265f2af21a admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:22:19.534999 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82aef70-15fe-4571-9a78-0a7641f0deb4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=164da7d4776d4e9f8cf6497aa784d312, user_id=52ed99737fd240789bb682fd9bf11be3, role_id=d0b9768f4d014766b9326387f2e11f34)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:22:19.538361 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82aef70-15fe-4571-9a78-0a7641f0deb4 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:22:19.593898 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 808/1603] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 01:22:19 2022] PUT /identity/v3/projects/53d2295ada114ea3b6b5001c4c197f35/users/efaf57b6482149ca86a9bba26d7654f2/roles/d0b9768f4d014766b9326387f2e11f34 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:22:19.595167 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a95bd59-547d-4c17-a8e2-0cd68f7be3c1 None None] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:19.595453 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a95bd59-547d-4c17-a8e2-0cd68f7be3c1 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:19.595726 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a95bd59-547d-4c17-a8e2-0cd68f7be3c1 None None] PATH_INFO: `/` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:19.597165 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 809/1604] 10.222.0.20 () {54 vars in 962 bytes} [Wed Aug 31 01:22:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:22:19.597849 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a95bd59-547d-4c17-a8e2-0cd68f7be3c1 None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:22:19.609554 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 796/1605] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 01:22:19 2022] PUT /identity/v3/projects/164da7d4776d4e9f8cf6497aa784d312/users/52ed99737fd240789bb682fd9bf11be3/roles/d0b9768f4d014766b9326387f2e11f34 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:22:19.610429 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82aef70-15fe-4571-9a78-0a7641f0deb4 admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:22:19.643676 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:22:19.678366 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a95bd59-547d-4c17-a8e2-0cd68f7be3c1 None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:22:19.680164 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82aef70-15fe-4571-9a78-0a7641f0deb4 admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:22:19.683734 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e1e8cc-5888-428d-8114-4ad42581beb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:22:19.685016 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e1e8cc-5888-428d-8114-4ad42581beb3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:19.685519 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf24069f-56ff-41f2-ac91-98b9ca278a79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:22:19.686213 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf24069f-56ff-41f2-ac91-98b9ca278a79 admin admin] REQUEST_METHOD: `PUT` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:19.686335 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e1e8cc-5888-428d-8114-4ad42581beb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:19.686497 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf24069f-56ff-41f2-ac91-98b9ca278a79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:19.686665 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e1e8cc-5888-428d-8114-4ad42581beb3 admin admin] PATH_INFO: `/v3/projects/d58414b1dd2342d58cc7665aee50b86b/users/71d0408df86f4be0ac59cc80575a14e2/roles/d0b9768f4d014766b9326387f2e11f34` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:19.687401 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf24069f-56ff-41f2-ac91-98b9ca278a79 admin admin] PATH_INFO: `/v3/projects/f8c9b1e4e8e246a58d474ae79c32fac3/users/965994e73e4641379fe77c7790233c1f/roles/d0b9768f4d014766b9326387f2e11f34` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:19.703731 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf24069f-56ff-41f2-ac91-98b9ca278a79 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f8c9b1e4e8e246a58d474ae79c32fac3, user_id=965994e73e4641379fe77c7790233c1f, role_id=d0b9768f4d014766b9326387f2e11f34)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:22:19.705293 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e1e8cc-5888-428d-8114-4ad42581beb3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d58414b1dd2342d58cc7665aee50b86b, user_id=71d0408df86f4be0ac59cc80575a14e2, role_id=d0b9768f4d014766b9326387f2e11f34)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:22:19.707354 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf24069f-56ff-41f2-ac91-98b9ca278a79 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:22:19.708427 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e1e8cc-5888-428d-8114-4ad42581beb3 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:22:19.763088 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 797/1606] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 01:22:19 2022] PUT /identity/v3/projects/f8c9b1e4e8e246a58d474ae79c32fac3/users/965994e73e4641379fe77c7790233c1f/roles/d0b9768f4d014766b9326387f2e11f34 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:22:19.764105 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf24069f-56ff-41f2-ac91-98b9ca278a79 admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:22:19.817070 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 810/1607] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 01:22:19 2022] PUT /identity/v3/projects/d58414b1dd2342d58cc7665aee50b86b/users/71d0408df86f4be0ac59cc80575a14e2/roles/d0b9768f4d014766b9326387f2e11f34 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:22:19.817912 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e1e8cc-5888-428d-8114-4ad42581beb3 admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:22:19.826367 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf24069f-56ff-41f2-ac91-98b9ca278a79 admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:22:19.874911 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:22:19.878651 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e1e8cc-5888-428d-8114-4ad42581beb3 admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:22:19.883134 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead35ce5-5b34-46b1-9754-0d515f8d2337 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0817c3320d5d41d8bba0550c8ae8dd60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:22:19.883751 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead35ce5-5b34-46b1-9754-0d515f8d2337 service neutron] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:19.884027 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead35ce5-5b34-46b1-9754-0d515f8d2337 service neutron] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:19.884300 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead35ce5-5b34-46b1-9754-0d515f8d2337 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:19.885598 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d68f20-52fc-44bb-8fd5-b3676a9d990c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:22:19.886272 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d68f20-52fc-44bb-8fd5-b3676a9d990c admin admin] REQUEST_METHOD: `PUT` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:19.886574 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d68f20-52fc-44bb-8fd5-b3676a9d990c admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:19.886842 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d68f20-52fc-44bb-8fd5-b3676a9d990c admin admin] PATH_INFO: `/v3/projects/4b5fd32565144659916a7686f464afef/users/7af6bd9048e14548b909ed120265ad76/roles/d0b9768f4d014766b9326387f2e11f34` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:19.893263 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead35ce5-5b34-46b1-9754-0d515f8d2337 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:22:19.895974 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead35ce5-5b34-46b1-9754-0d515f8d2337 service neutron] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:22:19.904940 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 811/1608] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:22:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:22:19.906205 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a18719-2ee9-4da3-a153-5c3f7becbd2b None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:19.906591 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a18719-2ee9-4da3-a153-5c3f7becbd2b None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:19.906834 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d68f20-52fc-44bb-8fd5-b3676a9d990c admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b5fd32565144659916a7686f464afef, user_id=7af6bd9048e14548b909ed120265ad76, role_id=d0b9768f4d014766b9326387f2e11f34)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:22:19.907072 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a18719-2ee9-4da3-a153-5c3f7becbd2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:19.909652 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d68f20-52fc-44bb-8fd5-b3676a9d990c admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:22:19.935042 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-64a18719-2ee9-4da3-a153-5c3f7becbd2b None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:22:19.936275 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-64a18719-2ee9-4da3-a153-5c3f7becbd2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:22:19.944181 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 812/1609] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:22:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:22:19.945558 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb59edb-ff53-45f6-beee-7d855ec33c3c None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:19.945838 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb59edb-ff53-45f6-beee-7d855ec33c3c None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:19.946133 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb59edb-ff53-45f6-beee-7d855ec33c3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:19.984466 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-acb59edb-ff53-45f6-beee-7d855ec33c3c None None] MFA Rules not processed for user `8744c9de803048f583d0317d01f6a540`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:22:19.985746 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-acb59edb-ff53-45f6-beee-7d855ec33c3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:22:20.003860 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 798/1610] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 01:22:19 2022] PUT /identity/v3/projects/4b5fd32565144659916a7686f464afef/users/7af6bd9048e14548b909ed120265ad76/roles/d0b9768f4d014766b9326387f2e11f34 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:22:20.003860 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaea6b9c-c2b2-4582-896a-4139761ff404 None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:20.003860 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaea6b9c-c2b2-4582-896a-4139761ff404 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:20.003860 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaea6b9c-c2b2-4582-896a-4139761ff404 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:20.026113 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-eaea6b9c-c2b2-4582-896a-4139761ff404 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:22:20.026113 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-eaea6b9c-c2b2-4582-896a-4139761ff404 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:22:20.064608 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 813/1611] 10.222.0.20 () {58 vars in 1059 bytes} [Wed Aug 31 01:22:19 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:22:20.066369 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7aa072c-2ea5-4187-944b-a9c498008229 None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:20.066811 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7aa072c-2ea5-4187-944b-a9c498008229 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:20.067393 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7aa072c-2ea5-4187-944b-a9c498008229 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:20.077450 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:22:20.099996 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 799/1612] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:22:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:22:20.102230 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2f4357-f723-4fe1-a719-f3ff38cfe434 None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:20.102527 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2f4357-f723-4fe1-a719-f3ff38cfe434 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:20.102964 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2f4357-f723-4fe1-a719-f3ff38cfe434 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:20.104608 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-f7aa072c-2ea5-4187-944b-a9c498008229 None None] MFA Rules not processed for user `965994e73e4641379fe77c7790233c1f`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:22:20.106133 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-f7aa072c-2ea5-4187-944b-a9c498008229 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:22:20.126215 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-3e2f4357-f723-4fe1-a719-f3ff38cfe434 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:22:20.127595 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-3e2f4357-f723-4fe1-a719-f3ff38cfe434 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:22:20.135485 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 800/1613] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:22:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:22:20.136356 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2f4357-f723-4fe1-a719-f3ff38cfe434 None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:22:20.177228 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 814/1614] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:22:20 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:22:20.178560 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf31e3f8-1312-436d-a4fb-d3a0611e9171 None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:20.178854 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf31e3f8-1312-436d-a4fb-d3a0611e9171 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:20.179131 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf31e3f8-1312-436d-a4fb-d3a0611e9171 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:20.204390 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2f4357-f723-4fe1-a719-f3ff38cfe434 None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:22:20.208827 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a7d661-c117-4842-9451-7af18f3d793b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0817c3320d5d41d8bba0550c8ae8dd60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:22:20.209441 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a7d661-c117-4842-9451-7af18f3d793b service neutron] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:20.210491 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a7d661-c117-4842-9451-7af18f3d793b service neutron] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:20.210772 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a7d661-c117-4842-9451-7af18f3d793b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:20.213379 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-cf31e3f8-1312-436d-a4fb-d3a0611e9171 None None] MFA Rules not processed for user `7af6bd9048e14548b909ed120265ad76`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:22:20.214837 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-cf31e3f8-1312-436d-a4fb-d3a0611e9171 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:22:20.221302 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a7d661-c117-4842-9451-7af18f3d793b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:22:20.227743 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a7d661-c117-4842-9451-7af18f3d793b service neutron] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:22:20.233888 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 801/1615] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:22:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:22:20.235086 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a7d661-c117-4842-9451-7af18f3d793b service neutron] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:22:20.246649 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a7d661-c117-4842-9451-7af18f3d793b service neutron] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:22:20.255402 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a6bfab-4145-4a32-b6e6-e2c8f37db1de service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8744c9de803048f583d0317d01f6a540', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:22:20.256136 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a6bfab-4145-4a32-b6e6-e2c8f37db1de service nova] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:20.257051 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a6bfab-4145-4a32-b6e6-e2c8f37db1de service nova] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:20.257757 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a6bfab-4145-4a32-b6e6-e2c8f37db1de service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:20.261618 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:22:20.292891 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 815/1616] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:22:20 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:22:20.296611 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf31e3f8-1312-436d-a4fb-d3a0611e9171 None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:22:20.301995 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-cf31e3f8-1312-436d-a4fb-d3a0611e9171 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:22:20.324239 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf31e3f8-1312-436d-a4fb-d3a0611e9171 None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:22:20.328412 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896fa68e-f115-4c9a-b424-9528a08b8bb9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0817c3320d5d41d8bba0550c8ae8dd60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:22:20.329387 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896fa68e-f115-4c9a-b424-9528a08b8bb9 service neutron] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:20.329729 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896fa68e-f115-4c9a-b424-9528a08b8bb9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:20.329946 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896fa68e-f115-4c9a-b424-9528a08b8bb9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:20.341748 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a6bfab-4145-4a32-b6e6-e2c8f37db1de service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:22:20.344755 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a6bfab-4145-4a32-b6e6-e2c8f37db1de service nova] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:22:20.355408 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896fa68e-f115-4c9a-b424-9528a08b8bb9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:22:20.356904 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896fa68e-f115-4c9a-b424-9528a08b8bb9 service neutron] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:22:20.372808 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 816/1617] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:22:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:22:20.373868 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896fa68e-f115-4c9a-b424-9528a08b8bb9 service neutron] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:22:20.384812 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 802/1618] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:22:20 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:22:20.388788 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896fa68e-f115-4c9a-b424-9528a08b8bb9 service neutron] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:22:20.388922 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a6bfab-4145-4a32-b6e6-e2c8f37db1de service nova] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:22:20.390958 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227f15f5-1a16-4622-a5e6-a7e6074f057d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0817c3320d5d41d8bba0550c8ae8dd60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:22:20.392946 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227f15f5-1a16-4622-a5e6-a7e6074f057d service neutron] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:20.393593 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227f15f5-1a16-4622-a5e6-a7e6074f057d service neutron] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:20.393969 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227f15f5-1a16-4622-a5e6-a7e6074f057d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:20.408380 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227f15f5-1a16-4622-a5e6-a7e6074f057d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:22:20.410807 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227f15f5-1a16-4622-a5e6-a7e6074f057d service neutron] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:22:20.413133 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a6bfab-4145-4a32-b6e6-e2c8f37db1de service nova] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:22:20.418777 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d15c2d-8953-4c99-9436-6e9aa7252e0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:22:20.419451 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d15c2d-8953-4c99-9436-6e9aa7252e0c admin admin] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:20.419741 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d15c2d-8953-4c99-9436-6e9aa7252e0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:20.420342 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d15c2d-8953-4c99-9436-6e9aa7252e0c admin admin] PATH_INFO: `/v3/projects` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:20.421032 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d15c2d-8953-4c99-9436-6e9aa7252e0c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:22:20.423022 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d15c2d-8953-4c99-9436-6e9aa7252e0c admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:22:20.425917 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 817/1619] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:22:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:22:20.427557 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227f15f5-1a16-4622-a5e6-a7e6074f057d service neutron] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:22:20.440151 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227f15f5-1a16-4622-a5e6-a7e6074f057d service neutron] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:22:20.447655 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c46caa-2ac2-4f29-be22-d5eef534e5b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:22:20.448397 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c46caa-2ac2-4f29-be22-d5eef534e5b2 admin admin] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:20.448855 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c46caa-2ac2-4f29-be22-d5eef534e5b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:20.449237 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c46caa-2ac2-4f29-be22-d5eef534e5b2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:20.450142 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c46caa-2ac2-4f29-be22-d5eef534e5b2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:22:20.452156 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c46caa-2ac2-4f29-be22-d5eef534e5b2 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:22:20.591010 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 803/1620] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 01:22:20 2022] POST /identity/v3/projects => generated 367 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:22:20.593654 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d15c2d-8953-4c99-9436-6e9aa7252e0c admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:22:20.595301 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 818/1621] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 01:22:20 2022] POST /identity/v3/projects => generated 379 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:22:20.596232 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c46caa-2ac2-4f29-be22-d5eef534e5b2 admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:22:20.699078 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:22:20.706821 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d15c2d-8953-4c99-9436-6e9aa7252e0c admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:22:20.709868 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c46caa-2ac2-4f29-be22-d5eef534e5b2 admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:22:20.712397 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0392591-89f0-4b12-a22c-16997926c88a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:22:20.713121 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0392591-89f0-4b12-a22c-16997926c88a admin admin] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:20.714750 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0392591-89f0-4b12-a22c-16997926c88a admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:20.715057 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0392591-89f0-4b12-a22c-16997926c88a admin admin] PATH_INFO: `/v3/users` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:20.715705 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0392591-89f0-4b12-a22c-16997926c88a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:22:20.717840 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0392591-89f0-4b12-a22c-16997926c88a admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:22:20.718300 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760a8e3c-7bed-4902-b4c3-696dbf252c78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:22:20.719642 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760a8e3c-7bed-4902-b4c3-696dbf252c78 admin admin] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:20.720277 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760a8e3c-7bed-4902-b4c3-696dbf252c78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:20.720844 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760a8e3c-7bed-4902-b4c3-696dbf252c78 admin admin] PATH_INFO: `/v3/users` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:20.721771 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760a8e3c-7bed-4902-b4c3-696dbf252c78 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:22:20.724636 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760a8e3c-7bed-4902-b4c3-696dbf252c78 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:22:20.825706 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 804/1622] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 01:22:20 2022] POST /identity/v3/users => generated 296 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:22:20.827365 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ffa490-cb37-40e5-a0cd-e7908b15b982 None None] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:20.827823 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ffa490-cb37-40e5-a0cd-e7908b15b982 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:20.828108 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ffa490-cb37-40e5-a0cd-e7908b15b982 None None] PATH_INFO: `/` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:20.829965 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 805/1623] 10.222.0.20 () {54 vars in 912 bytes} [Wed Aug 31 01:22:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:22:20.832471 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ffa490-cb37-40e5-a0cd-e7908b15b982 None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:22:20.846370 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ffa490-cb37-40e5-a0cd-e7908b15b982 None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:22:20.858492 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 819/1624] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 01:22:20 2022] POST /identity/v3/users => generated 290 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:22:20.862042 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ed8ac0-c69a-4d2a-a210-56a686942830 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:22:20.862452 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648cedc2-50f3-4aa2-aacb-249c6ce972c0 None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:20.862749 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648cedc2-50f3-4aa2-aacb-249c6ce972c0 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:20.863030 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648cedc2-50f3-4aa2-aacb-249c6ce972c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:20.863657 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ed8ac0-c69a-4d2a-a210-56a686942830 admin admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:20.864164 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ed8ac0-c69a-4d2a-a210-56a686942830 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:20.864668 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ed8ac0-c69a-4d2a-a210-56a686942830 admin admin] PATH_INFO: `/v3/roles` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:20.866957 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ed8ac0-c69a-4d2a-a210-56a686942830 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:22:20.868994 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ed8ac0-c69a-4d2a-a210-56a686942830 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:22:20.884985 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 806/1625] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:22:20 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:22:20.887150 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ed8ac0-c69a-4d2a-a210-56a686942830 admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:22:20.896251 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-648cedc2-50f3-4aa2-aacb-249c6ce972c0 None None] MFA Rules not processed for user `8744c9de803048f583d0317d01f6a540`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:22:20.896251 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-648cedc2-50f3-4aa2-aacb-249c6ce972c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:22:20.906864 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ed8ac0-c69a-4d2a-a210-56a686942830 admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:22:20.913725 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4926921b-ed23-4c74-ad19-60a90e4939c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:22:20.914805 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4926921b-ed23-4c74-ad19-60a90e4939c9 admin admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:20.915289 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4926921b-ed23-4c74-ad19-60a90e4939c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:20.915711 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4926921b-ed23-4c74-ad19-60a90e4939c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:20.916684 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4926921b-ed23-4c74-ad19-60a90e4939c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:22:20.921598 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4926921b-ed23-4c74-ad19-60a90e4939c9 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:22:20.931152 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 807/1626] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:22:20 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:22:20.932320 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4926921b-ed23-4c74-ad19-60a90e4939c9 admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:22:20.948559 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4926921b-ed23-4c74-ad19-60a90e4939c9 admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:22:20.954538 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4164c9-8737-43e7-9d33-39804b769ad2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:22:20.955540 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4164c9-8737-43e7-9d33-39804b769ad2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:20.955878 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4164c9-8737-43e7-9d33-39804b769ad2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:20.956250 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4164c9-8737-43e7-9d33-39804b769ad2 admin admin] PATH_INFO: `/v3/projects/2ab36940693d4ab185eb56876959d34f/users/9e23f4ccbd244889ae9c02fd7802901c/roles/d009411496b8452b88e77ffb8db015a7` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:20.985078 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4164c9-8737-43e7-9d33-39804b769ad2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ab36940693d4ab185eb56876959d34f, user_id=9e23f4ccbd244889ae9c02fd7802901c, role_id=d009411496b8452b88e77ffb8db015a7)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:22:20.989513 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4164c9-8737-43e7-9d33-39804b769ad2 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:22:20.992693 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 820/1627] 10.222.0.20 () {58 vars in 1009 bytes} [Wed Aug 31 01:22:20 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:22:20.993676 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648cedc2-50f3-4aa2-aacb-249c6ce972c0 None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:22:21.009483 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648cedc2-50f3-4aa2-aacb-249c6ce972c0 None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:22:21.014447 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88abc81e-a733-44e5-b2d8-7f1816ed1b18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:22:21.015125 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88abc81e-a733-44e5-b2d8-7f1816ed1b18 admin admin] REQUEST_METHOD: `PUT` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:21.015406 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88abc81e-a733-44e5-b2d8-7f1816ed1b18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:21.015684 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88abc81e-a733-44e5-b2d8-7f1816ed1b18 admin admin] PATH_INFO: `/v3/projects/68d95954bb524dbebf8fda826fc566bd/users/081a8e62ca474049a547c698276ded7a/roles/d009411496b8452b88e77ffb8db015a7` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:21.028030 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:22:21.030995 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88abc81e-a733-44e5-b2d8-7f1816ed1b18 admin admin] RBAC: Authorizing `identity:create_grant(project_id=68d95954bb524dbebf8fda826fc566bd, user_id=081a8e62ca474049a547c698276ded7a, role_id=d009411496b8452b88e77ffb8db015a7)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:22:21.033688 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88abc81e-a733-44e5-b2d8-7f1816ed1b18 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:22:21.095300 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 808/1628] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 01:22:20 2022] PUT /identity/v3/projects/2ab36940693d4ab185eb56876959d34f/users/9e23f4ccbd244889ae9c02fd7802901c/roles/d009411496b8452b88e77ffb8db015a7 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:22:21.098359 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4164c9-8737-43e7-9d33-39804b769ad2 admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:22:21.126839 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 821/1629] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 01:22:20 2022] PUT /identity/v3/projects/68d95954bb524dbebf8fda826fc566bd/users/081a8e62ca474049a547c698276ded7a/roles/d009411496b8452b88e77ffb8db015a7 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:22:21.127853 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88abc81e-a733-44e5-b2d8-7f1816ed1b18 admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:22:21.174187 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4164c9-8737-43e7-9d33-39804b769ad2 admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:22:21.238502 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e4da59-b550-4a1c-b0c9-4d84cd1a1975 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5c8fc69c518412ba1aad61a9f4f558c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:22:21.241759 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e4da59-b550-4a1c-b0c9-4d84cd1a1975 service cinder] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:21.241759 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e4da59-b550-4a1c-b0c9-4d84cd1a1975 service cinder] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:21.241759 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e4da59-b550-4a1c-b0c9-4d84cd1a1975 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:21.261990 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88abc81e-a733-44e5-b2d8-7f1816ed1b18 admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:22:21.270449 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95333add-a9e9-4d04-83d4-38df4e529bdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:22:21.270449 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95333add-a9e9-4d04-83d4-38df4e529bdc admin admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:21.270449 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95333add-a9e9-4d04-83d4-38df4e529bdc admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:21.270449 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95333add-a9e9-4d04-83d4-38df4e529bdc admin admin] PATH_INFO: `/v3/roles` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:21.271492 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95333add-a9e9-4d04-83d4-38df4e529bdc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:22:21.274918 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95333add-a9e9-4d04-83d4-38df4e529bdc admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:22:21.299130 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 822/1630] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:22:21 2022] GET /identity/v3/roles => generated 1420 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:22:21.299130 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95333add-a9e9-4d04-83d4-38df4e529bdc admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:22:21.321604 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95333add-a9e9-4d04-83d4-38df4e529bdc admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:22:21.331691 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e061593-d812-4204-9aff-1af0f328119f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:22:21.333826 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e061593-d812-4204-9aff-1af0f328119f admin admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:21.333826 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e061593-d812-4204-9aff-1af0f328119f admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:21.333826 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e061593-d812-4204-9aff-1af0f328119f admin admin] PATH_INFO: `/v3/roles` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:21.333826 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e061593-d812-4204-9aff-1af0f328119f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:22:21.335659 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e061593-d812-4204-9aff-1af0f328119f admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:22:21.346774 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 823/1631] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:22:21 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:22:21.347588 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e061593-d812-4204-9aff-1af0f328119f admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:22:21.360609 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e061593-d812-4204-9aff-1af0f328119f admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:22:21.368344 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e4e863-14c5-412e-841a-f8347a04ce51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:22:21.373992 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e4e863-14c5-412e-841a-f8347a04ce51 admin admin] REQUEST_METHOD: `PUT` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:21.373992 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e4e863-14c5-412e-841a-f8347a04ce51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:21.373992 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e4e863-14c5-412e-841a-f8347a04ce51 admin admin] PATH_INFO: `/v3/projects/2ab36940693d4ab185eb56876959d34f/users/9e23f4ccbd244889ae9c02fd7802901c/roles/d009411496b8452b88e77ffb8db015a7` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:21.373992 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e4e863-14c5-412e-841a-f8347a04ce51 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ab36940693d4ab185eb56876959d34f, user_id=9e23f4ccbd244889ae9c02fd7802901c, role_id=d009411496b8452b88e77ffb8db015a7)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:22:21.374348 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:22:21.375600 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e4e863-14c5-412e-841a-f8347a04ce51 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:22:21.377984 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e4da59-b550-4a1c-b0c9-4d84cd1a1975 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:22:21.380925 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e4da59-b550-4a1c-b0c9-4d84cd1a1975 service cinder] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:22:21.389395 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 824/1632] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 01:22:21 2022] PUT /identity/v3/projects/2ab36940693d4ab185eb56876959d34f/users/9e23f4ccbd244889ae9c02fd7802901c/roles/d009411496b8452b88e77ffb8db015a7 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:22:21.390189 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e4e863-14c5-412e-841a-f8347a04ce51 admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:22:21.401988 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 809/1633] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:22:21 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:22:21.403644 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9a0e22-2655-4616-8081-fca3f3ec5f9e None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:21.403996 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9a0e22-2655-4616-8081-fca3f3ec5f9e None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:21.404256 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9a0e22-2655-4616-8081-fca3f3ec5f9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:21.439057 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-9c9a0e22-2655-4616-8081-fca3f3ec5f9e None None] MFA Rules not processed for user `9e23f4ccbd244889ae9c02fd7802901c`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:22:21.440517 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-9c9a0e22-2655-4616-8081-fca3f3ec5f9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:22:21.546740 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 810/1634] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:22:21 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:22:21.580540 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e4e863-14c5-412e-841a-f8347a04ce51 admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:22:21.591490 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bef3e05-e617-4cbe-8b3a-c0b07005ecf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:22:21.592178 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bef3e05-e617-4cbe-8b3a-c0b07005ecf0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:21.592516 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bef3e05-e617-4cbe-8b3a-c0b07005ecf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:21.592814 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bef3e05-e617-4cbe-8b3a-c0b07005ecf0 admin admin] PATH_INFO: `/v3/projects/68d95954bb524dbebf8fda826fc566bd/users/081a8e62ca474049a547c698276ded7a/roles/d009411496b8452b88e77ffb8db015a7` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:21.595002 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bef3e05-e617-4cbe-8b3a-c0b07005ecf0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=68d95954bb524dbebf8fda826fc566bd, user_id=081a8e62ca474049a547c698276ded7a, role_id=d009411496b8452b88e77ffb8db015a7)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:22:21.597972 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bef3e05-e617-4cbe-8b3a-c0b07005ecf0 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:22:21.608877 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9a0e22-2655-4616-8081-fca3f3ec5f9e None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:22:21.635494 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 825/1635] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 01:22:21 2022] PUT /identity/v3/projects/68d95954bb524dbebf8fda826fc566bd/users/081a8e62ca474049a547c698276ded7a/roles/d009411496b8452b88e77ffb8db015a7 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:22:21.637387 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc27ee42-1a43-449d-bed0-f694c0c28d69 None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:21.637685 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc27ee42-1a43-449d-bed0-f694c0c28d69 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:21.638025 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc27ee42-1a43-449d-bed0-f694c0c28d69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:21.674865 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-fc27ee42-1a43-449d-bed0-f694c0c28d69 None None] MFA Rules not processed for user `081a8e62ca474049a547c698276ded7a`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:22:21.676626 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-fc27ee42-1a43-449d-bed0-f694c0c28d69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:22:21.787058 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:22:21.787308 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9a0e22-2655-4616-8081-fca3f3ec5f9e None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:22:21.844567 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 826/1636] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:22:21 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:22:21.890846 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18df92f5-0f78-4423-af0d-80e8f4a9297a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5c8fc69c518412ba1aad61a9f4f558c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:22:21.891735 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18df92f5-0f78-4423-af0d-80e8f4a9297a service cinder] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:21.892210 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18df92f5-0f78-4423-af0d-80e8f4a9297a service cinder] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:21.892667 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18df92f5-0f78-4423-af0d-80e8f4a9297a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:21.954091 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d56bfb9-1acd-4534-8de7-cb7278afbd5b None None] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:21.954877 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d56bfb9-1acd-4534-8de7-cb7278afbd5b None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:21.955163 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d56bfb9-1acd-4534-8de7-cb7278afbd5b None None] PATH_INFO: `/` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:21.956774 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 827/1637] 10.222.0.20 () {54 vars in 962 bytes} [Wed Aug 31 01:22:21 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 01:22:21.965211 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6001a76c-4f9a-4dc5-9f54-74534c1c32d9 None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:21.965700 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6001a76c-4f9a-4dc5-9f54-74534c1c32d9 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:21.966049 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6001a76c-4f9a-4dc5-9f54-74534c1c32d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:21.995284 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18df92f5-0f78-4423-af0d-80e8f4a9297a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:22:21.997317 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18df92f5-0f78-4423-af0d-80e8f4a9297a service cinder] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:22:22.001095 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-6001a76c-4f9a-4dc5-9f54-74534c1c32d9 None None] MFA Rules not processed for user `8744c9de803048f583d0317d01f6a540`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:22:22.001095 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-6001a76c-4f9a-4dc5-9f54-74534c1c32d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:22:22.010570 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 811/1638] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:22:21 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 01:22:22.101366 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 828/1639] 10.222.0.20 () {58 vars in 1059 bytes} [Wed Aug 31 01:22:21 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 01:22:22.107888 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4e1c16-5705-4421-9b76-9d88e9945b09 None None] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:22.108183 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4e1c16-5705-4421-9b76-9d88e9945b09 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:22.108463 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4e1c16-5705-4421-9b76-9d88e9945b09 None None] PATH_INFO: `/` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:22.110483 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 812/1640] 10.222.0.20 () {54 vars in 912 bytes} [Wed Aug 31 01:22:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:22:22.119652 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f063901-3a76-4a5c-8683-1a96456d3a44 None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:22.119652 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f063901-3a76-4a5c-8683-1a96456d3a44 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:22.119652 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f063901-3a76-4a5c-8683-1a96456d3a44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:22.144013 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-3f063901-3a76-4a5c-8683-1a96456d3a44 None None] MFA Rules not processed for user `8744c9de803048f583d0317d01f6a540`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:22:22.146880 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-3f063901-3a76-4a5c-8683-1a96456d3a44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:22:22.158367 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 829/1641] 10.222.0.20 () {58 vars in 1009 bytes} [Wed Aug 31 01:22:22 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:22:22.164790 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:22:22.185004 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4e1c16-5705-4421-9b76-9d88e9945b09 None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:22:22.197891 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4e1c16-5705-4421-9b76-9d88e9945b09 None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:22:22.203853 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb08148-9923-4402-8fc6-b686e4c2943a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5c8fc69c518412ba1aad61a9f4f558c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:22:22.206251 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb08148-9923-4402-8fc6-b686e4c2943a service cinder] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:22.206894 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb08148-9923-4402-8fc6-b686e4c2943a service cinder] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:22.207009 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb08148-9923-4402-8fc6-b686e4c2943a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:22.225878 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb08148-9923-4402-8fc6-b686e4c2943a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:22:22.229898 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb08148-9923-4402-8fc6-b686e4c2943a service cinder] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:22:22.258654 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f063901-3a76-4a5c-8683-1a96456d3a44 None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:22:22.261858 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 813/1642] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:22:22 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:22:22.276314 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f063901-3a76-4a5c-8683-1a96456d3a44 None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:22:22.282528 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4195b32b-c548-4701-8824-033d353462c2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5c8fc69c518412ba1aad61a9f4f558c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:22:22.289793 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4195b32b-c548-4701-8824-033d353462c2 service cinder] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:22.290141 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4195b32b-c548-4701-8824-033d353462c2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:22.290609 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4195b32b-c548-4701-8824-033d353462c2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:22.303456 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4195b32b-c548-4701-8824-033d353462c2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:22:22.304987 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4195b32b-c548-4701-8824-033d353462c2 service cinder] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:22:22.320804 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 830/1643] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:22:22 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 01:22:22.542920 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb08148-9923-4402-8fc6-b686e4c2943a service cinder] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:22:22.552898 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb08148-9923-4402-8fc6-b686e4c2943a service cinder] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:22:22.557738 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd528faf-581b-4900-80f7-0c2737b158a4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5c8fc69c518412ba1aad61a9f4f558c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:22:22.558530 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd528faf-581b-4900-80f7-0c2737b158a4 service cinder] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:22.558993 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd528faf-581b-4900-80f7-0c2737b158a4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:22.559424 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd528faf-581b-4900-80f7-0c2737b158a4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:22.656009 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd528faf-581b-4900-80f7-0c2737b158a4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:22:22.657896 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd528faf-581b-4900-80f7-0c2737b158a4 service cinder] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:22:22.683776 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 814/1644] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:22:22 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:22:23.638953 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4195b32b-c548-4701-8824-033d353462c2 service cinder] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:22:23.649303 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4195b32b-c548-4701-8824-033d353462c2 service cinder] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:22:23.655091 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a96796-6908-406d-8566-4d1328344638 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5c8fc69c518412ba1aad61a9f4f558c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:22:23.655836 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a96796-6908-406d-8566-4d1328344638 service cinder] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:23.656349 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a96796-6908-406d-8566-4d1328344638 service cinder] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:23.656643 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a96796-6908-406d-8566-4d1328344638 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:23.733679 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a96796-6908-406d-8566-4d1328344638 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:22:23.736541 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a96796-6908-406d-8566-4d1328344638 service cinder] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:22:23.774706 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 831/1645] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:22:23 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:22:28.855866 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b109f8a8-a601-47e3-8bec-a78911e42e5a None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:28.856338 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b109f8a8-a601-47e3-8bec-a78911e42e5a None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:28.856420 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b109f8a8-a601-47e3-8bec-a78911e42e5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:28.918768 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-b109f8a8-a601-47e3-8bec-a78911e42e5a None None] MFA Rules not processed for user `b7d058c2b40343a6a3e2e46f6b2bab0e`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:22:28.920254 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-b109f8a8-a601-47e3-8bec-a78911e42e5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:22:29.023346 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 815/1646] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:22:28 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:22:29.037983 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae78991-b1c0-4cf9-901e-a90927d2e7cb None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:29.038452 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae78991-b1c0-4cf9-901e-a90927d2e7cb None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:29.038786 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae78991-b1c0-4cf9-901e-a90927d2e7cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:29.066913 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-eae78991-b1c0-4cf9-901e-a90927d2e7cb None None] MFA Rules not processed for user `b7d058c2b40343a6a3e2e46f6b2bab0e`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:22:29.068666 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-eae78991-b1c0-4cf9-901e-a90927d2e7cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:22:29.095269 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 832/1647] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:22:29 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:22:29.106009 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b109f8a8-a601-47e3-8bec-a78911e42e5a None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:22:29.126364 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b109f8a8-a601-47e3-8bec-a78911e42e5a None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:22:29.145039 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf25b4c-2843-413a-b4f5-8ab0a1f80317 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8744c9de803048f583d0317d01f6a540', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:22:29.146119 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf25b4c-2843-413a-b4f5-8ab0a1f80317 service nova] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:29.146560 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf25b4c-2843-413a-b4f5-8ab0a1f80317 service nova] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:29.147499 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf25b4c-2843-413a-b4f5-8ab0a1f80317 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:29.160782 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf25b4c-2843-413a-b4f5-8ab0a1f80317 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:22:29.165465 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf25b4c-2843-413a-b4f5-8ab0a1f80317 service nova] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:22:29.219502 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 816/1648] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:22:29 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:22:29.440097 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae78991-b1c0-4cf9-901e-a90927d2e7cb None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:22:29.566734 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:22:29.598326 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae78991-b1c0-4cf9-901e-a90927d2e7cb None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:22:29.612230 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9b4d94-4bd3-466f-b45c-f58089199c84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:22:29.612230 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9b4d94-4bd3-466f-b45c-f58089199c84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:29.612230 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9b4d94-4bd3-466f-b45c-f58089199c84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:29.612230 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9b4d94-4bd3-466f-b45c-f58089199c84 admin admin] PATH_INFO: `/v3/users/965994e73e4641379fe77c7790233c1f` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:29.613427 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9b4d94-4bd3-466f-b45c-f58089199c84 admin admin] RBAC: Authorizing `identity:delete_user(user_id=965994e73e4641379fe77c7790233c1f)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:22:29.622879 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9b4d94-4bd3-466f-b45c-f58089199c84 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:22:29.871849 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-3a9b4d94-4bd3-466f-b45c-f58089199c84 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '965994e73e4641379fe77c7790233c1f'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:22:29.897563 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-3a9b4d94-4bd3-466f-b45c-f58089199c84 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '965994e73e4641379fe77c7790233c1f'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:22:29.899662 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-3a9b4d94-4bd3-466f-b45c-f58089199c84 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '965994e73e4641379fe77c7790233c1f'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:22:29.918474 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-3a9b4d94-4bd3-466f-b45c-f58089199c84 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '965994e73e4641379fe77c7790233c1f'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:22:29.993310 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-3a9b4d94-4bd3-466f-b45c-f58089199c84 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '965994e73e4641379fe77c7790233c1f'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:22:30.000055 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 833/1649] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 01:22:29 2022] DELETE /identity/v3/users/965994e73e4641379fe77c7790233c1f => generated 0 bytes in 572 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:22:30.004659 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c498360f-c924-48cd-8705-f1ab552ec9a1 None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:30.005177 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c498360f-c924-48cd-8705-f1ab552ec9a1 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:30.005253 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c498360f-c924-48cd-8705-f1ab552ec9a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:30.049966 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-c498360f-c924-48cd-8705-f1ab552ec9a1 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:22:30.051490 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-c498360f-c924-48cd-8705-f1ab552ec9a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:22:30.172413 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:22:30.206263 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a96c88-1cf1-4b0d-9b51-72904327c5fc None None] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:30.206717 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a96c88-1cf1-4b0d-9b51-72904327c5fc None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:30.206810 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a96c88-1cf1-4b0d-9b51-72904327c5fc None None] PATH_INFO: `/` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:30.215549 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 834/1650] 10.222.0.20 () {54 vars in 912 bytes} [Wed Aug 31 01:22:30 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:22:30.220684 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 817/1651] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:22:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:22:30.226387 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c778376c-b7f2-4141-b6b7-5358f2ab0d1f None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:30.226683 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c778376c-b7f2-4141-b6b7-5358f2ab0d1f None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:30.226966 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c778376c-b7f2-4141-b6b7-5358f2ab0d1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:30.270963 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c498360f-c924-48cd-8705-f1ab552ec9a1 None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:22:30.274788 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-c498360f-c924-48cd-8705-f1ab552ec9a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:22:30.326642 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-c778376c-b7f2-4141-b6b7-5358f2ab0d1f None None] MFA Rules not processed for user `0817c3320d5d41d8bba0550c8ae8dd60`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:22:30.328011 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-c778376c-b7f2-4141-b6b7-5358f2ab0d1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:22:30.381890 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c498360f-c924-48cd-8705-f1ab552ec9a1 None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:22:30.388836 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf392599-5422-4c9a-b803-369137e105f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0817c3320d5d41d8bba0550c8ae8dd60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:22:30.390139 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf392599-5422-4c9a-b803-369137e105f4 service neutron] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:30.390139 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf392599-5422-4c9a-b803-369137e105f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:30.390139 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf392599-5422-4c9a-b803-369137e105f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:30.404132 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf392599-5422-4c9a-b803-369137e105f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:22:30.406121 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf392599-5422-4c9a-b803-369137e105f4 service neutron] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:22:30.422951 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 835/1652] 10.222.0.20 () {58 vars in 1009 bytes} [Wed Aug 31 01:22:30 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:22:30.436930 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 818/1653] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:22:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:22:30.449201 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c778376c-b7f2-4141-b6b7-5358f2ab0d1f None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:22:30.462718 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c778376c-b7f2-4141-b6b7-5358f2ab0d1f None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:22:30.468202 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72667781-bca7-4547-b4b9-33b7e1a33f99 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0817c3320d5d41d8bba0550c8ae8dd60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:22:30.469247 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72667781-bca7-4547-b4b9-33b7e1a33f99 service neutron] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:30.469379 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72667781-bca7-4547-b4b9-33b7e1a33f99 service neutron] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:30.469843 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72667781-bca7-4547-b4b9-33b7e1a33f99 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:30.493233 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72667781-bca7-4547-b4b9-33b7e1a33f99 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:22:30.495792 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72667781-bca7-4547-b4b9-33b7e1a33f99 service neutron] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:22:30.526802 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 836/1654] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:22:30 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 78 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:22:31.309730 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf392599-5422-4c9a-b803-369137e105f4 service neutron] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:22:31.311211 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-bf392599-5422-4c9a-b803-369137e105f4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:22:31.338287 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf392599-5422-4c9a-b803-369137e105f4 service neutron] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:22:31.351872 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb517218-050a-413f-ac85-45d7de403024 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:22:31.352596 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb517218-050a-413f-ac85-45d7de403024 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:31.352931 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb517218-050a-413f-ac85-45d7de403024 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:31.353216 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb517218-050a-413f-ac85-45d7de403024 admin admin] PATH_INFO: `/v3/projects/f8c9b1e4e8e246a58d474ae79c32fac3` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:31.356376 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb517218-050a-413f-ac85-45d7de403024 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f8c9b1e4e8e246a58d474ae79c32fac3)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:22:31.358654 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb517218-050a-413f-ac85-45d7de403024 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:22:31.557386 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-bb517218-050a-413f-ac85-45d7de403024 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f8c9b1e4e8e246a58d474ae79c32fac3', 'request_id': 'req-bb517218-050a-413f-ac85-45d7de403024'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:22:31.566177 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-bb517218-050a-413f-ac85-45d7de403024 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f8c9b1e4e8e246a58d474ae79c32fac3', 'request_id': 'req-bb517218-050a-413f-ac85-45d7de403024'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:22:31.583356 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-bb517218-050a-413f-ac85-45d7de403024 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f8c9b1e4e8e246a58d474ae79c32fac3', 'request_id': 'req-bb517218-050a-413f-ac85-45d7de403024'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:22:31.689117 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-bb517218-050a-413f-ac85-45d7de403024 admin admin] The token cache is being invalidate because project f8c9b1e4e8e246a58d474ae79c32fac3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81749) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 01:22:31.689637 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-bb517218-050a-413f-ac85-45d7de403024 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:22:31.690744 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-bb517218-050a-413f-ac85-45d7de403024 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:22:31.693966 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 819/1655] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 01:22:31 2022] DELETE /identity/v3/projects/f8c9b1e4e8e246a58d474ae79c32fac3 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:22:31.698189 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72667781-bca7-4547-b4b9-33b7e1a33f99 service neutron] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:22:31.729939 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72667781-bca7-4547-b4b9-33b7e1a33f99 service neutron] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:22:31.812402 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb517218-050a-413f-ac85-45d7de403024 admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:22:31.814244 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-bb517218-050a-413f-ac85-45d7de403024 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:22:31.895249 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:22:31.926293 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2d589e-09d8-4bdd-81a4-8f5b0b74854a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:22:31.927063 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2d589e-09d8-4bdd-81a4-8f5b0b74854a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:31.927668 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2d589e-09d8-4bdd-81a4-8f5b0b74854a admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:31.928822 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2d589e-09d8-4bdd-81a4-8f5b0b74854a admin admin] PATH_INFO: `/v3/users/081a8e62ca474049a547c698276ded7a` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:31.930578 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2d589e-09d8-4bdd-81a4-8f5b0b74854a admin admin] RBAC: Authorizing `identity:delete_user(user_id=081a8e62ca474049a547c698276ded7a)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:22:31.932842 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2d589e-09d8-4bdd-81a4-8f5b0b74854a admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:22:32.016568 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb517218-050a-413f-ac85-45d7de403024 admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:22:32.032417 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5b5e9d-8829-4aee-b786-30e18113e1cc service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dea0fe5b1d6b405cb47b95192580303b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:22:32.033301 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5b5e9d-8829-4aee-b786-30e18113e1cc service placement] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:32.033733 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5b5e9d-8829-4aee-b786-30e18113e1cc service placement] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:32.034285 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5b5e9d-8829-4aee-b786-30e18113e1cc service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:32.036979 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-9b5b5e9d-8829-4aee-b786-30e18113e1cc service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:22:32.058312 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5b5e9d-8829-4aee-b786-30e18113e1cc service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:22:32.060174 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5b5e9d-8829-4aee-b786-30e18113e1cc service placement] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:22:32.072418 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 820/1656] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:22:31 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 261 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 01:22:32.073641 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5b5e9d-8829-4aee-b786-30e18113e1cc service placement] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:22:32.075401 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-9b5b5e9d-8829-4aee-b786-30e18113e1cc service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:22:32.088225 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:22:32.098775 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5b5e9d-8829-4aee-b786-30e18113e1cc service placement] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:22:32.103800 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84052187-89f2-48ba-8bf9-6c68ebb24f4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:22:32.104450 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84052187-89f2-48ba-8bf9-6c68ebb24f4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:32.104760 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84052187-89f2-48ba-8bf9-6c68ebb24f4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:32.105077 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84052187-89f2-48ba-8bf9-6c68ebb24f4f admin admin] PATH_INFO: `/v3/users/8b73724e8e94402d8d18777f57a257e3` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:32.107676 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84052187-89f2-48ba-8bf9-6c68ebb24f4f admin admin] RBAC: Authorizing `identity:delete_user(user_id=8b73724e8e94402d8d18777f57a257e3)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:22:32.108649 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: WARNING py.warnings [None req-84052187-89f2-48ba-8bf9-6c68ebb24f4f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 31 01:22:32.108649 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: warnings.warn(msg) Aug 31 01:22:32.108649 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: Aug 31 01:22:32.110406 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84052187-89f2-48ba-8bf9-6c68ebb24f4f admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:22:32.170787 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-5e2d589e-09d8-4bdd-81a4-8f5b0b74854a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '081a8e62ca474049a547c698276ded7a'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:22:32.199633 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-5e2d589e-09d8-4bdd-81a4-8f5b0b74854a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '081a8e62ca474049a547c698276ded7a'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:22:32.201145 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-5e2d589e-09d8-4bdd-81a4-8f5b0b74854a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '081a8e62ca474049a547c698276ded7a'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:22:32.244536 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-5e2d589e-09d8-4bdd-81a4-8f5b0b74854a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '081a8e62ca474049a547c698276ded7a'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:22:32.322516 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-5e2d589e-09d8-4bdd-81a4-8f5b0b74854a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '081a8e62ca474049a547c698276ded7a'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:22:32.330870 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 837/1657] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 01:22:31 2022] DELETE /identity/v3/users/081a8e62ca474049a547c698276ded7a => generated 0 bytes in 633 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:22:32.386456 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-84052187-89f2-48ba-8bf9-6c68ebb24f4f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8b73724e8e94402d8d18777f57a257e3'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:22:32.477529 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-84052187-89f2-48ba-8bf9-6c68ebb24f4f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8b73724e8e94402d8d18777f57a257e3'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:22:32.478488 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-84052187-89f2-48ba-8bf9-6c68ebb24f4f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8b73724e8e94402d8d18777f57a257e3'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:22:32.502141 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-84052187-89f2-48ba-8bf9-6c68ebb24f4f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8b73724e8e94402d8d18777f57a257e3'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:22:32.503490 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-84052187-89f2-48ba-8bf9-6c68ebb24f4f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8b73724e8e94402d8d18777f57a257e3'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:22:32.582628 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 821/1658] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 01:22:32 2022] DELETE /identity/v3/users/8b73724e8e94402d8d18777f57a257e3 => generated 0 bytes in 509 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:22:32.588682 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37301b6e-1052-4c78-9f3f-0914d8df71ea None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:32.589237 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37301b6e-1052-4c78-9f3f-0914d8df71ea None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:32.589820 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37301b6e-1052-4c78-9f3f-0914d8df71ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:32.644879 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-37301b6e-1052-4c78-9f3f-0914d8df71ea None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:22:32.647250 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-37301b6e-1052-4c78-9f3f-0914d8df71ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:22:32.762080 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 838/1659] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:22:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:22:32.778389 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84052187-89f2-48ba-8bf9-6c68ebb24f4f admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:22:32.781160 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-84052187-89f2-48ba-8bf9-6c68ebb24f4f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:22:32.950342 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84052187-89f2-48ba-8bf9-6c68ebb24f4f admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:22:32.961927 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4f8c61-6884-4973-84f6-b15aeb66c240 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0817c3320d5d41d8bba0550c8ae8dd60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:22:32.963302 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4f8c61-6884-4973-84f6-b15aeb66c240 service neutron] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:32.964826 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4f8c61-6884-4973-84f6-b15aeb66c240 service neutron] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:32.964826 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4f8c61-6884-4973-84f6-b15aeb66c240 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:33.001209 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4f8c61-6884-4973-84f6-b15aeb66c240 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:22:33.004931 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4f8c61-6884-4973-84f6-b15aeb66c240 service neutron] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:22:33.029790 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 822/1660] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:22:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:22:34.415617 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37301b6e-1052-4c78-9f3f-0914d8df71ea None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:22:34.417543 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-37301b6e-1052-4c78-9f3f-0914d8df71ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:22:34.491046 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37301b6e-1052-4c78-9f3f-0914d8df71ea None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:22:34.517445 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e707da9-9e65-47f8-936c-a4f94196131a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:22:34.520459 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e707da9-9e65-47f8-936c-a4f94196131a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:34.521040 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e707da9-9e65-47f8-936c-a4f94196131a admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:34.521814 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e707da9-9e65-47f8-936c-a4f94196131a admin admin] PATH_INFO: `/v3/projects/314a7f3b9a1c48e98ff114f51e82ff01` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:34.523321 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e707da9-9e65-47f8-936c-a4f94196131a admin admin] RBAC: Authorizing `identity:delete_project(project_id=314a7f3b9a1c48e98ff114f51e82ff01)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:22:34.524633 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: WARNING py.warnings [None req-4e707da9-9e65-47f8-936c-a4f94196131a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 31 01:22:34.524633 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: warnings.warn(msg) Aug 31 01:22:34.524633 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: Aug 31 01:22:34.526611 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e707da9-9e65-47f8-936c-a4f94196131a admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:22:34.736546 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-4e707da9-9e65-47f8-936c-a4f94196131a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '314a7f3b9a1c48e98ff114f51e82ff01', 'request_id': 'req-4e707da9-9e65-47f8-936c-a4f94196131a'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:22:34.753626 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-4e707da9-9e65-47f8-936c-a4f94196131a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '314a7f3b9a1c48e98ff114f51e82ff01', 'request_id': 'req-4e707da9-9e65-47f8-936c-a4f94196131a'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:22:34.761900 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:22:34.830741 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4f8c61-6884-4973-84f6-b15aeb66c240 service neutron] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:22:34.833105 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-2c4f8c61-6884-4973-84f6-b15aeb66c240 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:22:34.838682 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-4e707da9-9e65-47f8-936c-a4f94196131a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '314a7f3b9a1c48e98ff114f51e82ff01', 'request_id': 'req-4e707da9-9e65-47f8-936c-a4f94196131a'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:22:34.893471 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4f8c61-6884-4973-84f6-b15aeb66c240 service neutron] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:22:34.907862 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-4e707da9-9e65-47f8-936c-a4f94196131a admin admin] The token cache is being invalidate because project 314a7f3b9a1c48e98ff114f51e82ff01 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81750) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 01:22:34.909347 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-4e707da9-9e65-47f8-936c-a4f94196131a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:22:34.911329 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-4e707da9-9e65-47f8-936c-a4f94196131a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:22:34.916504 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 839/1661] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 01:22:34 2022] DELETE /identity/v3/projects/314a7f3b9a1c48e98ff114f51e82ff01 => generated 0 bytes in 501 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:22:34.921666 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f12dec4-05cd-4d79-8f8a-452d361a33a2 None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:34.922250 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f12dec4-05cd-4d79-8f8a-452d361a33a2 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:34.922752 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f12dec4-05cd-4d79-8f8a-452d361a33a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:34.976552 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-0f12dec4-05cd-4d79-8f8a-452d361a33a2 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:22:34.984416 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-0f12dec4-05cd-4d79-8f8a-452d361a33a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:22:35.055863 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c7ef1d-8900-4cda-9dd3-edf16383d167 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:22:35.058375 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c7ef1d-8900-4cda-9dd3-edf16383d167 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:35.058922 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c7ef1d-8900-4cda-9dd3-edf16383d167 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:35.060786 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c7ef1d-8900-4cda-9dd3-edf16383d167 admin admin] PATH_INFO: `/v3/projects/68d95954bb524dbebf8fda826fc566bd` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:35.063095 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c7ef1d-8900-4cda-9dd3-edf16383d167 admin admin] RBAC: Authorizing `identity:delete_project(project_id=68d95954bb524dbebf8fda826fc566bd)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:22:35.068242 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c7ef1d-8900-4cda-9dd3-edf16383d167 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:22:35.176082 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:22:35.177215 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 840/1662] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:22:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:22:35.178840 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d709484-22ae-461a-b290-54d52f540130 None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:35.179112 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d709484-22ae-461a-b290-54d52f540130 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:35.179394 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d709484-22ae-461a-b290-54d52f540130 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:35.222076 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-3d709484-22ae-461a-b290-54d52f540130 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:22:35.223330 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-3d709484-22ae-461a-b290-54d52f540130 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:22:35.250612 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 841/1663] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:22:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:22:35.251780 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d709484-22ae-461a-b290-54d52f540130 None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:22:35.273811 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d709484-22ae-461a-b290-54d52f540130 None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:22:35.325067 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-f7c7ef1d-8900-4cda-9dd3-edf16383d167 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '68d95954bb524dbebf8fda826fc566bd', 'request_id': 'req-f7c7ef1d-8900-4cda-9dd3-edf16383d167'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:22:35.345355 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-f7c7ef1d-8900-4cda-9dd3-edf16383d167 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '68d95954bb524dbebf8fda826fc566bd', 'request_id': 'req-f7c7ef1d-8900-4cda-9dd3-edf16383d167'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:22:35.355680 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-f7c7ef1d-8900-4cda-9dd3-edf16383d167 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '68d95954bb524dbebf8fda826fc566bd', 'request_id': 'req-f7c7ef1d-8900-4cda-9dd3-edf16383d167'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:22:35.415921 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e709bf-9d3c-468f-856d-b93d75efe3a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:22:35.418926 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e709bf-9d3c-468f-856d-b93d75efe3a0 admin admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:35.419302 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e709bf-9d3c-468f-856d-b93d75efe3a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:35.419690 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e709bf-9d3c-468f-856d-b93d75efe3a0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:35.420580 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e709bf-9d3c-468f-856d-b93d75efe3a0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:22:35.422230 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e709bf-9d3c-468f-856d-b93d75efe3a0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:22:35.424397 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e709bf-9d3c-468f-856d-b93d75efe3a0 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:22:35.497623 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-f7c7ef1d-8900-4cda-9dd3-edf16383d167 admin admin] The token cache is being invalidate because project 68d95954bb524dbebf8fda826fc566bd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81749) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 01:22:35.498628 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-f7c7ef1d-8900-4cda-9dd3-edf16383d167 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:22:35.499332 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-f7c7ef1d-8900-4cda-9dd3-edf16383d167 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:22:35.508673 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 823/1664] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 01:22:34 2022] DELETE /identity/v3/projects/68d95954bb524dbebf8fda826fc566bd => generated 0 bytes in 675 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 01:22:35.512787 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bd4dc1-a9e7-47d0-8006-fb8907107d86 None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:35.513077 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bd4dc1-a9e7-47d0-8006-fb8907107d86 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:35.513358 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bd4dc1-a9e7-47d0-8006-fb8907107d86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:35.519169 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 842/1665] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Aug 31 01:22:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:22:35.527863 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e709bf-9d3c-468f-856d-b93d75efe3a0 admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:22:35.531456 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-b8e709bf-9d3c-468f-856d-b93d75efe3a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:22:35.570170 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e709bf-9d3c-468f-856d-b93d75efe3a0 admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:22:35.581568 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5127ab-1904-46a2-92c0-99137998f817 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:22:35.582206 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5127ab-1904-46a2-92c0-99137998f817 admin admin] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:35.582479 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5127ab-1904-46a2-92c0-99137998f817 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:35.586021 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5127ab-1904-46a2-92c0-99137998f817 admin admin] PATH_INFO: `/v3/projects` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:35.586021 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5127ab-1904-46a2-92c0-99137998f817 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:22:35.586210 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-88bd4dc1-a9e7-47d0-8006-fb8907107d86 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:22:35.593305 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-88bd4dc1-a9e7-47d0-8006-fb8907107d86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:22:35.605679 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5127ab-1904-46a2-92c0-99137998f817 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:22:35.647514 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 824/1666] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:22:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:22:35.653782 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8651c4-2845-4005-a585-6cf0436ea21a None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:35.654080 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8651c4-2845-4005-a585-6cf0436ea21a None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:35.655892 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8651c4-2845-4005-a585-6cf0436ea21a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:35.717762 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-aa8651c4-2845-4005-a585-6cf0436ea21a None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:22:35.718993 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-aa8651c4-2845-4005-a585-6cf0436ea21a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:22:35.751826 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 825/1667] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:22:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:22:35.752901 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8651c4-2845-4005-a585-6cf0436ea21a None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:22:35.783570 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 843/1668] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 01:22:35 2022] POST /identity/v3/projects => generated 403 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:22:35.784859 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5127ab-1904-46a2-92c0-99137998f817 admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:22:35.786108 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8651c4-2845-4005-a585-6cf0436ea21a None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:22:35.928526 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820d0ac9-bf67-44fc-bb4f-95be69b63726 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:22:35.929906 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820d0ac9-bf67-44fc-bb4f-95be69b63726 admin admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:35.930336 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820d0ac9-bf67-44fc-bb4f-95be69b63726 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:35.930756 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820d0ac9-bf67-44fc-bb4f-95be69b63726 admin admin] PATH_INFO: `/v3/domains` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:35.931866 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820d0ac9-bf67-44fc-bb4f-95be69b63726 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:22:35.932322 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820d0ac9-bf67-44fc-bb4f-95be69b63726 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:22:35.934449 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820d0ac9-bf67-44fc-bb4f-95be69b63726 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:22:35.946758 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5127ab-1904-46a2-92c0-99137998f817 admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:22:35.950365 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 826/1669] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Aug 31 01:22:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 01:22:35.959625 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820d0ac9-bf67-44fc-bb4f-95be69b63726 admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:22:35.962692 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e63cdd6-0c2e-4769-9081-390d8d8211b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:22:35.965248 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e63cdd6-0c2e-4769-9081-390d8d8211b6 admin admin] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:35.965248 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e63cdd6-0c2e-4769-9081-390d8d8211b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:35.965636 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e63cdd6-0c2e-4769-9081-390d8d8211b6 admin admin] PATH_INFO: `/v3/users` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:35.966300 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e63cdd6-0c2e-4769-9081-390d8d8211b6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:22:35.968476 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e63cdd6-0c2e-4769-9081-390d8d8211b6 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:22:36.012347 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820d0ac9-bf67-44fc-bb4f-95be69b63726 admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:22:36.030722 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e1baf8-f48c-44cf-95f3-c4b050832b71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:22:36.032600 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e1baf8-f48c-44cf-95f3-c4b050832b71 admin admin] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:36.033198 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e1baf8-f48c-44cf-95f3-c4b050832b71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:36.034654 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e1baf8-f48c-44cf-95f3-c4b050832b71 admin admin] PATH_INFO: `/v3/projects` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:36.035550 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e1baf8-f48c-44cf-95f3-c4b050832b71 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:22:36.043519 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e1baf8-f48c-44cf-95f3-c4b050832b71 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:22:36.130325 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 844/1670] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 01:22:35 2022] POST /identity/v3/users => generated 302 bytes in 346 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 01:22:36.136898 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e63cdd6-0c2e-4769-9081-390d8d8211b6 admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:22:36.163136 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e63cdd6-0c2e-4769-9081-390d8d8211b6 admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:22:36.172966 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774bfb3b-5a14-4e59-9234-df2cace09607 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:22:36.173433 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774bfb3b-5a14-4e59-9234-df2cace09607 admin admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:36.173723 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774bfb3b-5a14-4e59-9234-df2cace09607 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:36.174006 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774bfb3b-5a14-4e59-9234-df2cace09607 admin admin] PATH_INFO: `/v3/roles` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:36.175599 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774bfb3b-5a14-4e59-9234-df2cace09607 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:22:36.182876 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774bfb3b-5a14-4e59-9234-df2cace09607 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:22:36.192163 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 827/1671] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 01:22:35 2022] POST /identity/v3/projects => generated 387 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 01:22:36.196405 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e1baf8-f48c-44cf-95f3-c4b050832b71 admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:22:36.211582 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 845/1672] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:22:36 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 01:22:36.220686 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774bfb3b-5a14-4e59-9234-df2cace09607 admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:22:36.272000 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774bfb3b-5a14-4e59-9234-df2cace09607 admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:22:36.391269 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e1baf8-f48c-44cf-95f3-c4b050832b71 admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:22:36.397940 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348b2ae2-1eff-41cd-8079-2f19d36ebba6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:22:36.398947 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348b2ae2-1eff-41cd-8079-2f19d36ebba6 admin admin] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:36.399490 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348b2ae2-1eff-41cd-8079-2f19d36ebba6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:36.399912 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348b2ae2-1eff-41cd-8079-2f19d36ebba6 admin admin] PATH_INFO: `/v3/users` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:36.400817 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348b2ae2-1eff-41cd-8079-2f19d36ebba6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:22:36.403498 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348b2ae2-1eff-41cd-8079-2f19d36ebba6 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:22:36.462784 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa83508f-2822-432c-a465-60009728a80a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:22:36.464692 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa83508f-2822-432c-a465-60009728a80a admin admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:36.465316 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa83508f-2822-432c-a465-60009728a80a admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:36.466026 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa83508f-2822-432c-a465-60009728a80a admin admin] PATH_INFO: `/v3/roles` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:36.467126 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa83508f-2822-432c-a465-60009728a80a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:22:36.473208 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa83508f-2822-432c-a465-60009728a80a admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:22:36.510223 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 846/1673] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:22:36 2022] GET /identity/v3/roles => generated 1420 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:22:36.511940 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852b7809-6059-4f50-9b92-18d0b0d71953 None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:36.512451 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852b7809-6059-4f50-9b92-18d0b0d71953 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:36.513006 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852b7809-6059-4f50-9b92-18d0b0d71953 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:36.569225 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 828/1674] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 01:22:36 2022] POST /identity/v3/users => generated 294 bytes in 373 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:22:36.570392 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348b2ae2-1eff-41cd-8079-2f19d36ebba6 admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:22:36.620783 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348b2ae2-1eff-41cd-8079-2f19d36ebba6 admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:22:36.632202 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-852b7809-6059-4f50-9b92-18d0b0d71953 None None] MFA Rules not processed for user `71d0408df86f4be0ac59cc80575a14e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:22:36.662701 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-852b7809-6059-4f50-9b92-18d0b0d71953 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:22:36.666653 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693b98b9-7239-4cc2-865d-e63fc83cc7cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:22:36.667812 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693b98b9-7239-4cc2-865d-e63fc83cc7cb admin admin] REQUEST_METHOD: `PUT` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:36.668241 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693b98b9-7239-4cc2-865d-e63fc83cc7cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:36.668667 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693b98b9-7239-4cc2-865d-e63fc83cc7cb admin admin] PATH_INFO: `/v3/projects/d4010fc9e21948adb150a09384956241/users/63cc85f4861948e79c90753f4ce2b4e9/roles/d0b9768f4d014766b9326387f2e11f34` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:36.728371 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:22:36.756788 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693b98b9-7239-4cc2-865d-e63fc83cc7cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=d4010fc9e21948adb150a09384956241, user_id=63cc85f4861948e79c90753f4ce2b4e9, role_id=d0b9768f4d014766b9326387f2e11f34)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:22:36.759575 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693b98b9-7239-4cc2-865d-e63fc83cc7cb admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:22:36.906173 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 829/1675] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 01:22:36 2022] PUT /identity/v3/projects/d4010fc9e21948adb150a09384956241/users/63cc85f4861948e79c90753f4ce2b4e9/roles/d0b9768f4d014766b9326387f2e11f34 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:22:36.907362 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693b98b9-7239-4cc2-865d-e63fc83cc7cb admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:22:36.940169 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 847/1676] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:22:36 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 429 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:22:36.942049 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9327466e-a031-42a0-9ef3-91fc55d6e4a5 None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:36.944571 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9327466e-a031-42a0-9ef3-91fc55d6e4a5 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:36.945052 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9327466e-a031-42a0-9ef3-91fc55d6e4a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:37.015957 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-9327466e-a031-42a0-9ef3-91fc55d6e4a5 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:22:37.017253 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-9327466e-a031-42a0-9ef3-91fc55d6e4a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:22:37.170583 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693b98b9-7239-4cc2-865d-e63fc83cc7cb admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:22:37.174364 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 848/1677] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:22:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:22:37.176062 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0baf12-fe11-4d2c-82b2-21ccfd3c1919 None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:37.184065 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0baf12-fe11-4d2c-82b2-21ccfd3c1919 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:37.184312 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0baf12-fe11-4d2c-82b2-21ccfd3c1919 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:37.202478 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5627d9fe-d171-472f-a629-d85e19e96a54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:22:37.202843 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5627d9fe-d171-472f-a629-d85e19e96a54 admin admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:37.205812 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:DELETE=7,keystone:SELECT=88,keystone:INSERT=5 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:22:37.207791 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5627d9fe-d171-472f-a629-d85e19e96a54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:37.208084 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5627d9fe-d171-472f-a629-d85e19e96a54 admin admin] PATH_INFO: `/v3/roles` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:37.209493 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5627d9fe-d171-472f-a629-d85e19e96a54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:22:37.211295 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5627d9fe-d171-472f-a629-d85e19e96a54 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:22:37.251587 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 830/1678] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:22:36 2022] GET /identity/v3/roles => generated 1420 bytes in 344 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:22:37.260580 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5627d9fe-d171-472f-a629-d85e19e96a54 admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:22:37.265328 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-5a0baf12-fe11-4d2c-82b2-21ccfd3c1919 None None] MFA Rules not processed for user `71d0408df86f4be0ac59cc80575a14e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:22:37.272427 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-5627d9fe-d171-472f-a629-d85e19e96a54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:22:37.272552 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-5a0baf12-fe11-4d2c-82b2-21ccfd3c1919 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:22:37.314741 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5627d9fe-d171-472f-a629-d85e19e96a54 admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:22:37.328081 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee9e228-51d0-45a1-a3d8-f457bca0160b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:22:37.336936 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee9e228-51d0-45a1-a3d8-f457bca0160b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:37.336936 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee9e228-51d0-45a1-a3d8-f457bca0160b admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:37.336936 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee9e228-51d0-45a1-a3d8-f457bca0160b admin admin] PATH_INFO: `/v3/users/000ebe06ff37469a81d0af9fb3e1ba28` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:37.339516 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee9e228-51d0-45a1-a3d8-f457bca0160b admin admin] RBAC: Authorizing `identity:delete_user(user_id=000ebe06ff37469a81d0af9fb3e1ba28)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:22:37.341529 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee9e228-51d0-45a1-a3d8-f457bca0160b admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:22:37.454632 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 849/1679] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:22:37 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:22:37.454632 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0baf12-fe11-4d2c-82b2-21ccfd3c1919 None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:22:37.454632 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-5a0baf12-fe11-4d2c-82b2-21ccfd3c1919 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:22:37.674642 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-eee9e228-51d0-45a1-a3d8-f457bca0160b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '000ebe06ff37469a81d0af9fb3e1ba28'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:22:37.731509 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0baf12-fe11-4d2c-82b2-21ccfd3c1919 None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:22:37.747720 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160e625b-0a3c-42f4-ba15-96e5216734bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0817c3320d5d41d8bba0550c8ae8dd60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:22:37.748367 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160e625b-0a3c-42f4-ba15-96e5216734bb service neutron] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:37.748651 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160e625b-0a3c-42f4-ba15-96e5216734bb service neutron] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:37.750488 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160e625b-0a3c-42f4-ba15-96e5216734bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:37.797524 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160e625b-0a3c-42f4-ba15-96e5216734bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:22:37.798987 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160e625b-0a3c-42f4-ba15-96e5216734bb service neutron] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:22:37.802314 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-eee9e228-51d0-45a1-a3d8-f457bca0160b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '000ebe06ff37469a81d0af9fb3e1ba28'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:22:37.808712 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-eee9e228-51d0-45a1-a3d8-f457bca0160b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '000ebe06ff37469a81d0af9fb3e1ba28'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:22:37.822781 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 850/1680] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:22:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:22:37.824372 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-eee9e228-51d0-45a1-a3d8-f457bca0160b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '000ebe06ff37469a81d0af9fb3e1ba28'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:22:37.827452 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-eee9e228-51d0-45a1-a3d8-f457bca0160b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '000ebe06ff37469a81d0af9fb3e1ba28'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:22:37.838682 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160e625b-0a3c-42f4-ba15-96e5216734bb service neutron] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:22:37.864418 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:22:37.877296 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160e625b-0a3c-42f4-ba15-96e5216734bb service neutron] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:22:37.878071 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 831/1681] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 01:22:37 2022] DELETE /identity/v3/users/000ebe06ff37469a81d0af9fb3e1ba28 => generated 0 bytes in 626 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:22:37.879295 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee9e228-51d0-45a1-a3d8-f457bca0160b admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:22:37.883613 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-eee9e228-51d0-45a1-a3d8-f457bca0160b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:22:38.020097 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4635244-89e9-480e-909e-e1d90d7a7a34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:22:38.021304 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4635244-89e9-480e-909e-e1d90d7a7a34 admin admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:38.021932 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4635244-89e9-480e-909e-e1d90d7a7a34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:38.022373 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4635244-89e9-480e-909e-e1d90d7a7a34 admin admin] PATH_INFO: `/v3/roles` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:38.023885 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4635244-89e9-480e-909e-e1d90d7a7a34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:22:38.026297 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4635244-89e9-480e-909e-e1d90d7a7a34 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:22:38.052261 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 851/1682] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:22:37 2022] GET /identity/v3/roles => generated 1420 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:22:38.057918 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981de588-8939-418d-8fed-912a9f54f398 None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:38.069874 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981de588-8939-418d-8fed-912a9f54f398 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:38.070451 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981de588-8939-418d-8fed-912a9f54f398 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:38.110058 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee9e228-51d0-45a1-a3d8-f457bca0160b admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:22:38.127152 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adf35ad-8996-446f-a302-51e808a52592 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8744c9de803048f583d0317d01f6a540', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:22:38.127989 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adf35ad-8996-446f-a302-51e808a52592 service nova] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:38.128526 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adf35ad-8996-446f-a302-51e808a52592 service nova] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:38.128951 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adf35ad-8996-446f-a302-51e808a52592 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:38.133790 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-9adf35ad-8996-446f-a302-51e808a52592 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:22:38.149416 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-981de588-8939-418d-8fed-912a9f54f398 None None] MFA Rules not processed for user `52ed99737fd240789bb682fd9bf11be3`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:22:38.150800 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-981de588-8939-418d-8fed-912a9f54f398 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:22:38.294977 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adf35ad-8996-446f-a302-51e808a52592 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:22:38.295526 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 852/1683] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:22:38 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:22:38.296717 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd00ae0-cc70-4328-950f-9229029d5e8c None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:38.297024 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd00ae0-cc70-4328-950f-9229029d5e8c None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:38.297307 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd00ae0-cc70-4328-950f-9229029d5e8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:38.298344 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adf35ad-8996-446f-a302-51e808a52592 service nova] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:22:38.332072 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 832/1684] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:22:37 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 453 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:22:38.347402 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99be5a8f-0068-4c8a-b048-17991c9e4515 None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:38.349992 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99be5a8f-0068-4c8a-b048-17991c9e4515 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:38.357090 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99be5a8f-0068-4c8a-b048-17991c9e4515 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:38.402715 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-99be5a8f-0068-4c8a-b048-17991c9e4515 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:22:38.404567 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-99be5a8f-0068-4c8a-b048-17991c9e4515 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:22:38.422140 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 833/1685] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:22:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:22:38.422140 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99be5a8f-0068-4c8a-b048-17991c9e4515 None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:22:38.423756 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-4bd00ae0-cc70-4328-950f-9229029d5e8c None None] MFA Rules not processed for user `b3efb4f9daac4a0ca05a36dd361f17e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:22:38.425656 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-99be5a8f-0068-4c8a-b048-17991c9e4515 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:22:38.425955 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-4bd00ae0-cc70-4328-950f-9229029d5e8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:22:38.446663 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99be5a8f-0068-4c8a-b048-17991c9e4515 None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:22:38.462823 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e29231b-b9fd-4b79-89c4-5dbc7c3cb0ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:22:38.463573 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e29231b-b9fd-4b79-89c4-5dbc7c3cb0ce admin admin] REQUEST_METHOD: `PUT` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:38.463858 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e29231b-b9fd-4b79-89c4-5dbc7c3cb0ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:38.465074 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e29231b-b9fd-4b79-89c4-5dbc7c3cb0ce admin admin] PATH_INFO: `/v3/projects/171940f7cda6497e93297c29bb1baae1/users/04a03327e5f342dc9dbf7a5d638a61c3/roles/d0b9768f4d014766b9326387f2e11f34` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:38.509890 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e29231b-b9fd-4b79-89c4-5dbc7c3cb0ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=171940f7cda6497e93297c29bb1baae1, user_id=04a03327e5f342dc9dbf7a5d638a61c3, role_id=d0b9768f4d014766b9326387f2e11f34)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:22:38.512785 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e29231b-b9fd-4b79-89c4-5dbc7c3cb0ce admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:22:38.596814 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 834/1686] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 01:22:38 2022] PUT /identity/v3/projects/171940f7cda6497e93297c29bb1baae1/users/04a03327e5f342dc9dbf7a5d638a61c3/roles/d0b9768f4d014766b9326387f2e11f34 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:22:38.596814 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e29231b-b9fd-4b79-89c4-5dbc7c3cb0ce admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:22:38.597298 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-7e29231b-b9fd-4b79-89c4-5dbc7c3cb0ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:22:38.683592 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 853/1687] 10.222.0.20 () {58 vars in 1009 bytes} [Wed Aug 31 01:22:38 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 387 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:22:38.684788 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd00ae0-cc70-4328-950f-9229029d5e8c None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:22:38.691239 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-4bd00ae0-cc70-4328-950f-9229029d5e8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:22:38.700426 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:22:38.747426 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:22:38.788199 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e29231b-b9fd-4b79-89c4-5dbc7c3cb0ce admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:22:38.795065 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46234e7-1e89-44f4-a235-18f3a580fb01 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8744c9de803048f583d0317d01f6a540', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:22:38.795808 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46234e7-1e89-44f4-a235-18f3a580fb01 service nova] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:38.796044 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46234e7-1e89-44f4-a235-18f3a580fb01 service nova] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:38.796323 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46234e7-1e89-44f4-a235-18f3a580fb01 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:38.871737 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd00ae0-cc70-4328-950f-9229029d5e8c None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:22:38.923760 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46234e7-1e89-44f4-a235-18f3a580fb01 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:22:38.925692 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46234e7-1e89-44f4-a235-18f3a580fb01 service nova] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:22:38.956579 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 835/1688] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:22:38 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:22:38.962413 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0a4084-dfc4-42a3-a746-d429a3305128 None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:38.965572 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0a4084-dfc4-42a3-a746-d429a3305128 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:38.966115 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0a4084-dfc4-42a3-a746-d429a3305128 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:39.026170 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0913a6a4-56d1-4f64-919a-3c3b3d1cd51b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0817c3320d5d41d8bba0550c8ae8dd60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:22:39.026901 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0913a6a4-56d1-4f64-919a-3c3b3d1cd51b service neutron] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:39.027261 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0913a6a4-56d1-4f64-919a-3c3b3d1cd51b service neutron] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:39.027612 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0913a6a4-56d1-4f64-919a-3c3b3d1cd51b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:39.061774 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:22:39.109888 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-2c0a4084-dfc4-42a3-a746-d429a3305128 None None] MFA Rules not processed for user `04a03327e5f342dc9dbf7a5d638a61c3`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:22:39.114722 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-2c0a4084-dfc4-42a3-a746-d429a3305128 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:22:39.228843 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0913a6a4-56d1-4f64-919a-3c3b3d1cd51b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:22:39.230658 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0913a6a4-56d1-4f64-919a-3c3b3d1cd51b service neutron] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:22:39.261905 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 854/1689] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:22:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 578 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:22:39.262900 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0913a6a4-56d1-4f64-919a-3c3b3d1cd51b service neutron] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:22:39.266868 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-0913a6a4-56d1-4f64-919a-3c3b3d1cd51b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:22:39.311967 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 836/1690] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:22:38 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 349 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:22:39.311967 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0a4084-dfc4-42a3-a746-d429a3305128 None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:22:39.332732 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0a4084-dfc4-42a3-a746-d429a3305128 None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:22:39.344391 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8eb78b8-1f87-4615-9d94-cbea1ac7dff1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:22:39.345955 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8eb78b8-1f87-4615-9d94-cbea1ac7dff1 admin admin] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:39.346897 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8eb78b8-1f87-4615-9d94-cbea1ac7dff1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:39.347202 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8eb78b8-1f87-4615-9d94-cbea1ac7dff1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:39.349820 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8eb78b8-1f87-4615-9d94-cbea1ac7dff1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:22:39.353477 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8eb78b8-1f87-4615-9d94-cbea1ac7dff1 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:22:39.480185 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 837/1691] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 01:22:39 2022] POST /identity/v3/projects => generated 387 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 01:22:39.481260 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8eb78b8-1f87-4615-9d94-cbea1ac7dff1 admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:22:39.483243 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-e8eb78b8-1f87-4615-9d94-cbea1ac7dff1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:22:39.519521 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0913a6a4-56d1-4f64-919a-3c3b3d1cd51b service neutron] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:22:39.605308 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da541635-da10-4a76-847b-fe8c18225afc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc1d010919ec40878ae829cc8dceec4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:22:39.606057 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da541635-da10-4a76-847b-fe8c18225afc service swift] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:39.606263 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da541635-da10-4a76-847b-fe8c18225afc service swift] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:39.606541 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da541635-da10-4a76-847b-fe8c18225afc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:39.625518 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8eb78b8-1f87-4615-9d94-cbea1ac7dff1 admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:22:39.635962 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb63f4c-92c7-41bf-b397-4a9ee265b5bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:22:39.636826 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb63f4c-92c7-41bf-b397-4a9ee265b5bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:39.637213 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb63f4c-92c7-41bf-b397-4a9ee265b5bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:39.637585 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb63f4c-92c7-41bf-b397-4a9ee265b5bd admin admin] PATH_INFO: `/v3/users/7af6bd9048e14548b909ed120265ad76` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:39.639870 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb63f4c-92c7-41bf-b397-4a9ee265b5bd admin admin] RBAC: Authorizing `identity:delete_user(user_id=7af6bd9048e14548b909ed120265ad76)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:22:39.641819 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb63f4c-92c7-41bf-b397-4a9ee265b5bd admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:22:39.668563 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:22:39.669020 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:22:39.712198 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da541635-da10-4a76-847b-fe8c18225afc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:22:39.712198 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da541635-da10-4a76-847b-fe8c18225afc service swift] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:22:39.722847 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 855/1692] 10.222.0.20 () {60 vars in 1384 bytes} [Wed Aug 31 01:22:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 460 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 01:22:39.723795 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f75d9d-95ce-4e50-89dd-dc4d399507f8 None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:39.725124 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f75d9d-95ce-4e50-89dd-dc4d399507f8 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:39.725124 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f75d9d-95ce-4e50-89dd-dc4d399507f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:39.774920 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-82f75d9d-95ce-4e50-89dd-dc4d399507f8 None None] MFA Rules not processed for user `efaf57b6482149ca86a9bba26d7654f2`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:22:39.781626 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-82f75d9d-95ce-4e50-89dd-dc4d399507f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:22:39.897727 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-9fb63f4c-92c7-41bf-b397-4a9ee265b5bd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7af6bd9048e14548b909ed120265ad76'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:22:39.991692 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-9fb63f4c-92c7-41bf-b397-4a9ee265b5bd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7af6bd9048e14548b909ed120265ad76'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:22:39.992598 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-9fb63f4c-92c7-41bf-b397-4a9ee265b5bd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7af6bd9048e14548b909ed120265ad76'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:22:40.001355 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-9fb63f4c-92c7-41bf-b397-4a9ee265b5bd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7af6bd9048e14548b909ed120265ad76'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:22:40.008844 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-9fb63f4c-92c7-41bf-b397-4a9ee265b5bd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7af6bd9048e14548b909ed120265ad76'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:22:40.016385 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 856/1693] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:22:39 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 293 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:22:40.018609 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f75d9d-95ce-4e50-89dd-dc4d399507f8 None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:22:40.066960 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f75d9d-95ce-4e50-89dd-dc4d399507f8 None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:22:40.073765 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e259ca65-80e2-4e8e-89a6-edb9e29c63a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:22:40.075585 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 838/1694] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 01:22:39 2022] DELETE /identity/v3/users/7af6bd9048e14548b909ed120265ad76 => generated 0 bytes in 595 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:22:40.075988 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e259ca65-80e2-4e8e-89a6-edb9e29c63a8 admin admin] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:40.076301 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e259ca65-80e2-4e8e-89a6-edb9e29c63a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:40.076586 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e259ca65-80e2-4e8e-89a6-edb9e29c63a8 admin admin] PATH_INFO: `/v3/users` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:40.077439 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e259ca65-80e2-4e8e-89a6-edb9e29c63a8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:22:40.081555 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e259ca65-80e2-4e8e-89a6-edb9e29c63a8 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:22:40.091495 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380e5ae8-6e60-4f6a-9e32-f28ca57c3a7c None None] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:40.092155 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380e5ae8-6e60-4f6a-9e32-f28ca57c3a7c None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:40.092630 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380e5ae8-6e60-4f6a-9e32-f28ca57c3a7c None None] PATH_INFO: `/` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:40.097526 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 839/1695] 10.222.0.20 () {54 vars in 912 bytes} [Wed Aug 31 01:22:40 2022] GET /identity => generated 270 bytes in 21 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:22:40.099074 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa235a95-5103-4ffc-b51c-41d891f66ea1 None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:40.101678 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa235a95-5103-4ffc-b51c-41d891f66ea1 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:40.102358 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa235a95-5103-4ffc-b51c-41d891f66ea1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:40.181924 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-fa235a95-5103-4ffc-b51c-41d891f66ea1 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:22:40.187103 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-fa235a95-5103-4ffc-b51c-41d891f66ea1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:22:40.272487 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 857/1696] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 01:22:40 2022] POST /identity/v3/users => generated 294 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:22:40.274773 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0327c86-5e14-4d57-af20-af29fca22936 None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:40.276127 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0327c86-5e14-4d57-af20-af29fca22936 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:40.283646 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0327c86-5e14-4d57-af20-af29fca22936 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:40.395195 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 840/1697] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:22:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 297 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:22:40.396818 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f766b783-28a8-4c19-bb08-724d68d9b8ac None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:40.397112 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f766b783-28a8-4c19-bb08-724d68d9b8ac None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:40.402029 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f766b783-28a8-4c19-bb08-724d68d9b8ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:40.406883 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-c0327c86-5e14-4d57-af20-af29fca22936 None None] MFA Rules not processed for user `0817c3320d5d41d8bba0550c8ae8dd60`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:22:40.409634 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-c0327c86-5e14-4d57-af20-af29fca22936 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:22:40.451360 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-f766b783-28a8-4c19-bb08-724d68d9b8ac None None] MFA Rules not processed for user `0817c3320d5d41d8bba0550c8ae8dd60`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:22:40.452581 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-f766b783-28a8-4c19-bb08-724d68d9b8ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:22:40.515439 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:22:40.579404 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 858/1698] 10.222.0.20 () {58 vars in 1009 bytes} [Wed Aug 31 01:22:40 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 306 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:22:40.582876 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0327c86-5e14-4d57-af20-af29fca22936 None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:22:40.583325 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 841/1699] 10.222.0.85 () {58 vars in 1015 bytes} [Wed Aug 31 01:22:40 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:22:40.584759 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f766b783-28a8-4c19-bb08-724d68d9b8ac None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:22:40.589906 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-f766b783-28a8-4c19-bb08-724d68d9b8ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:22:40.592797 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-c0327c86-5e14-4d57-af20-af29fca22936 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:22:40.618188 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f766b783-28a8-4c19-bb08-724d68d9b8ac None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:22:40.634510 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cefa21-ac77-42b2-a651-1aed8876cdf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:22:40.635147 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cefa21-ac77-42b2-a651-1aed8876cdf2 admin admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:40.635419 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cefa21-ac77-42b2-a651-1aed8876cdf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:40.635694 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cefa21-ac77-42b2-a651-1aed8876cdf2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:40.636364 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58cefa21-ac77-42b2-a651-1aed8876cdf2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:22:40.638037 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58cefa21-ac77-42b2-a651-1aed8876cdf2 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:22:40.666880 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 842/1700] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:22:40 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:22:40.669384 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cefa21-ac77-42b2-a651-1aed8876cdf2 admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:22:40.678477 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-58cefa21-ac77-42b2-a651-1aed8876cdf2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:22:40.702158 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cefa21-ac77-42b2-a651-1aed8876cdf2 admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:22:40.706938 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675860a7-2616-4f14-93e0-d69b46b9ccb9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0817c3320d5d41d8bba0550c8ae8dd60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:22:40.708046 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675860a7-2616-4f14-93e0-d69b46b9ccb9 service neutron] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:40.709143 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675860a7-2616-4f14-93e0-d69b46b9ccb9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:40.709622 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675860a7-2616-4f14-93e0-d69b46b9ccb9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:40.721634 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675860a7-2616-4f14-93e0-d69b46b9ccb9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:22:40.723565 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675860a7-2616-4f14-93e0-d69b46b9ccb9 service neutron] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:22:40.740190 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 843/1701] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:22:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:22:40.744841 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675860a7-2616-4f14-93e0-d69b46b9ccb9 service neutron] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:22:40.753957 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675860a7-2616-4f14-93e0-d69b46b9ccb9 service neutron] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:22:40.766910 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6aa1e59-b3f9-41b4-9fa6-36eec7836656 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0817c3320d5d41d8bba0550c8ae8dd60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:22:40.767485 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6aa1e59-b3f9-41b4-9fa6-36eec7836656 service neutron] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:40.767851 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6aa1e59-b3f9-41b4-9fa6-36eec7836656 service neutron] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:40.768058 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6aa1e59-b3f9-41b4-9fa6-36eec7836656 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:40.780517 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6aa1e59-b3f9-41b4-9fa6-36eec7836656 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:22:40.782049 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6aa1e59-b3f9-41b4-9fa6-36eec7836656 service neutron] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:22:40.791345 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 844/1702] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:22:40 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:22:40.792370 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6aa1e59-b3f9-41b4-9fa6-36eec7836656 service neutron] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:22:40.801211 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:22:40.802023 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6aa1e59-b3f9-41b4-9fa6-36eec7836656 service neutron] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:22:40.807796 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595e2b6e-f50d-4387-a982-5a7e92ff75ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0817c3320d5d41d8bba0550c8ae8dd60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:22:40.808482 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595e2b6e-f50d-4387-a982-5a7e92ff75ea service neutron] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:40.809015 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595e2b6e-f50d-4387-a982-5a7e92ff75ea service neutron] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:40.809294 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595e2b6e-f50d-4387-a982-5a7e92ff75ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:40.819669 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595e2b6e-f50d-4387-a982-5a7e92ff75ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:22:40.821587 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595e2b6e-f50d-4387-a982-5a7e92ff75ea service neutron] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:22:40.832346 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 845/1703] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:22:40 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:22:40.833414 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595e2b6e-f50d-4387-a982-5a7e92ff75ea service neutron] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:22:40.851421 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0327c86-5e14-4d57-af20-af29fca22936 None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:22:40.858267 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595e2b6e-f50d-4387-a982-5a7e92ff75ea service neutron] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:22:40.867543 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62ee25a-3084-4bb7-91a8-24f3f003d6c9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8744c9de803048f583d0317d01f6a540', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:22:40.868194 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62ee25a-3084-4bb7-91a8-24f3f003d6c9 service nova] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:40.868472 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62ee25a-3084-4bb7-91a8-24f3f003d6c9 service nova] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:40.868784 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62ee25a-3084-4bb7-91a8-24f3f003d6c9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:40.870924 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19090df-2971-44e7-a2d6-b4997ff07507 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:22:40.871589 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19090df-2971-44e7-a2d6-b4997ff07507 admin admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:40.871872 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19090df-2971-44e7-a2d6-b4997ff07507 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:40.872146 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19090df-2971-44e7-a2d6-b4997ff07507 admin admin] PATH_INFO: `/v3/roles` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:40.873158 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19090df-2971-44e7-a2d6-b4997ff07507 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:22:40.875514 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-a62ee25a-3084-4bb7-91a8-24f3f003d6c9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:22:40.879258 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19090df-2971-44e7-a2d6-b4997ff07507 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:22:40.900340 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 846/1704] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:22:40 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:22:40.901205 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19090df-2971-44e7-a2d6-b4997ff07507 admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:22:40.905705 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-d19090df-2971-44e7-a2d6-b4997ff07507 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:22:40.936340 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19090df-2971-44e7-a2d6-b4997ff07507 admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:22:40.956640 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc17b3c-78ce-410f-93b7-2f311fd94b1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:22:40.957534 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc17b3c-78ce-410f-93b7-2f311fd94b1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:40.958560 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc17b3c-78ce-410f-93b7-2f311fd94b1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:40.960132 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc17b3c-78ce-410f-93b7-2f311fd94b1b admin admin] PATH_INFO: `/v3/projects/5c58218b6d974ee6bd1a873fd2d5bb16` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:40.967848 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc17b3c-78ce-410f-93b7-2f311fd94b1b admin admin] RBAC: Authorizing `identity:delete_project(project_id=5c58218b6d974ee6bd1a873fd2d5bb16)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:22:40.967848 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc17b3c-78ce-410f-93b7-2f311fd94b1b admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:22:40.991999 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62ee25a-3084-4bb7-91a8-24f3f003d6c9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:22:40.994023 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62ee25a-3084-4bb7-91a8-24f3f003d6c9 service nova] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:22:41.017831 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 859/1705] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:22:40 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:22:41.019983 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62ee25a-3084-4bb7-91a8-24f3f003d6c9 service nova] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:22:41.048200 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62ee25a-3084-4bb7-91a8-24f3f003d6c9 service nova] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:22:41.056199 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94f3722-f95f-46a5-b9a6-56f671a0a630 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:22:41.057203 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94f3722-f95f-46a5-b9a6-56f671a0a630 admin admin] REQUEST_METHOD: `PUT` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:41.057660 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94f3722-f95f-46a5-b9a6-56f671a0a630 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:41.058105 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94f3722-f95f-46a5-b9a6-56f671a0a630 admin admin] PATH_INFO: `/v3/projects/9a29535d173c49c398e2bc713df03755/users/d9d2efb56e2b48a68317275a0a8c115e/roles/d0b9768f4d014766b9326387f2e11f34` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:41.076336 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94f3722-f95f-46a5-b9a6-56f671a0a630 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a29535d173c49c398e2bc713df03755, user_id=d9d2efb56e2b48a68317275a0a8c115e, role_id=d0b9768f4d014766b9326387f2e11f34)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:22:41.080555 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94f3722-f95f-46a5-b9a6-56f671a0a630 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:22:41.182937 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 860/1706] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 01:22:41 2022] PUT /identity/v3/projects/9a29535d173c49c398e2bc713df03755/users/d9d2efb56e2b48a68317275a0a8c115e/roles/d0b9768f4d014766b9326387f2e11f34 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:22:41.194676 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07cc96e-69d2-4581-8b1b-b60c654638fd None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:41.195520 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07cc96e-69d2-4581-8b1b-b60c654638fd None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:41.195520 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07cc96e-69d2-4581-8b1b-b60c654638fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:41.211438 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:22:41.242450 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-ddc17b3c-78ce-410f-93b7-2f311fd94b1b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5c58218b6d974ee6bd1a873fd2d5bb16', 'request_id': 'req-ddc17b3c-78ce-410f-93b7-2f311fd94b1b'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:22:41.251765 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-c07cc96e-69d2-4581-8b1b-b60c654638fd None None] MFA Rules not processed for user `d9d2efb56e2b48a68317275a0a8c115e`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:22:41.253040 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-c07cc96e-69d2-4581-8b1b-b60c654638fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:22:41.261135 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-ddc17b3c-78ce-410f-93b7-2f311fd94b1b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5c58218b6d974ee6bd1a873fd2d5bb16', 'request_id': 'req-ddc17b3c-78ce-410f-93b7-2f311fd94b1b'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:22:41.289074 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-ddc17b3c-78ce-410f-93b7-2f311fd94b1b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5c58218b6d974ee6bd1a873fd2d5bb16', 'request_id': 'req-ddc17b3c-78ce-410f-93b7-2f311fd94b1b'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:22:41.406063 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 861/1707] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:22:41 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:22:41.412900 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07cc96e-69d2-4581-8b1b-b60c654638fd None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:22:41.562353 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07cc96e-69d2-4581-8b1b-b60c654638fd None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:22:41.569128 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac1e1df-6394-4f60-9ab3-57549205924c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:22:41.569913 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac1e1df-6394-4f60-9ab3-57549205924c admin admin] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:41.570303 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac1e1df-6394-4f60-9ab3-57549205924c admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:41.570699 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac1e1df-6394-4f60-9ab3-57549205924c admin admin] PATH_INFO: `/v3/projects` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:41.571470 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac1e1df-6394-4f60-9ab3-57549205924c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:22:41.574853 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac1e1df-6394-4f60-9ab3-57549205924c admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:22:41.582708 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:22:41.768300 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-ddc17b3c-78ce-410f-93b7-2f311fd94b1b admin admin] The token cache is being invalidate because project 5c58218b6d974ee6bd1a873fd2d5bb16 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81749) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 01:22:41.769255 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-ddc17b3c-78ce-410f-93b7-2f311fd94b1b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:22:41.773385 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-ddc17b3c-78ce-410f-93b7-2f311fd94b1b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:22:41.775197 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 847/1708] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 01:22:40 2022] DELETE /identity/v3/projects/5c58218b6d974ee6bd1a873fd2d5bb16 => generated 0 bytes in 874 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:22:41.782369 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3572b68b-86e0-4d80-8781-bd9e277cdebc None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:41.782655 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3572b68b-86e0-4d80-8781-bd9e277cdebc None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:41.782936 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3572b68b-86e0-4d80-8781-bd9e277cdebc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:41.828551 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-3572b68b-86e0-4d80-8781-bd9e277cdebc None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:22:41.832447 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-3572b68b-86e0-4d80-8781-bd9e277cdebc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:22:41.856745 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 848/1709] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:22:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:22:41.867251 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29db030-022f-4790-810e-a4791cc5c5d2 None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:41.867251 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29db030-022f-4790-810e-a4791cc5c5d2 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:41.868003 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29db030-022f-4790-810e-a4791cc5c5d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:41.894436 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 862/1710] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 01:22:41 2022] POST /identity/v3/projects => generated 387 bytes in 482 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:22:41.899209 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac1e1df-6394-4f60-9ab3-57549205924c admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:22:41.902640 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-3ac1e1df-6394-4f60-9ab3-57549205924c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:22:41.958571 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-f29db030-022f-4790-810e-a4791cc5c5d2 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:22:41.967741 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-f29db030-022f-4790-810e-a4791cc5c5d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:22:42.178663 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 849/1711] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:22:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:22:42.182243 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29db030-022f-4790-810e-a4791cc5c5d2 None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:22:42.195719 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac1e1df-6394-4f60-9ab3-57549205924c admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:22:42.201613 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29db030-022f-4790-810e-a4791cc5c5d2 None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:22:42.221733 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648a4bd9-c958-41a6-ad2b-3695685f6d72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:22:42.222378 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648a4bd9-c958-41a6-ad2b-3695685f6d72 admin admin] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:42.222669 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648a4bd9-c958-41a6-ad2b-3695685f6d72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:42.222948 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648a4bd9-c958-41a6-ad2b-3695685f6d72 admin admin] PATH_INFO: `/v3/users` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:42.225014 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648a4bd9-c958-41a6-ad2b-3695685f6d72 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:22:42.234801 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648a4bd9-c958-41a6-ad2b-3695685f6d72 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:22:42.239668 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb250389-2e10-41a0-bd49-75c95b7f38d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:22:42.242530 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb250389-2e10-41a0-bd49-75c95b7f38d0 admin admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:42.243208 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb250389-2e10-41a0-bd49-75c95b7f38d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:42.244039 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb250389-2e10-41a0-bd49-75c95b7f38d0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:42.247267 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb250389-2e10-41a0-bd49-75c95b7f38d0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:22:42.247837 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb250389-2e10-41a0-bd49-75c95b7f38d0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:22:42.256990 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb250389-2e10-41a0-bd49-75c95b7f38d0 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:22:42.292803 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 850/1712] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Aug 31 01:22:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:22:42.297002 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb250389-2e10-41a0-bd49-75c95b7f38d0 admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:22:42.316151 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb250389-2e10-41a0-bd49-75c95b7f38d0 admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:22:42.331970 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cb1215-2193-4b4e-8368-5c0a953260d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:22:42.333621 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cb1215-2193-4b4e-8368-5c0a953260d3 admin admin] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:42.334053 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cb1215-2193-4b4e-8368-5c0a953260d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:42.334339 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cb1215-2193-4b4e-8368-5c0a953260d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:42.336323 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cb1215-2193-4b4e-8368-5c0a953260d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:22:42.344198 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cb1215-2193-4b4e-8368-5c0a953260d3 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:22:42.395583 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 863/1713] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 01:22:41 2022] POST /identity/v3/users => generated 300 bytes in 497 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:22:42.400838 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648a4bd9-c958-41a6-ad2b-3695685f6d72 admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:22:42.427029 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648a4bd9-c958-41a6-ad2b-3695685f6d72 admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:22:42.438394 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97668632-eda5-4580-a171-8469a9720888 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:22:42.441505 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97668632-eda5-4580-a171-8469a9720888 admin admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:42.448458 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97668632-eda5-4580-a171-8469a9720888 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:42.449002 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97668632-eda5-4580-a171-8469a9720888 admin admin] PATH_INFO: `/v3/roles` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:42.450348 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97668632-eda5-4580-a171-8469a9720888 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:22:42.454759 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97668632-eda5-4580-a171-8469a9720888 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:22:42.471429 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 864/1714] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:22:42 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 01:22:42.472908 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97668632-eda5-4580-a171-8469a9720888 admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:22:42.486286 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97668632-eda5-4580-a171-8469a9720888 admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:22:42.527457 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6fbb82-6219-4b1a-8d9f-acd1e474ad6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:22:42.527457 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6fbb82-6219-4b1a-8d9f-acd1e474ad6b admin admin] REQUEST_METHOD: `PUT` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:42.527457 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6fbb82-6219-4b1a-8d9f-acd1e474ad6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:42.527457 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6fbb82-6219-4b1a-8d9f-acd1e474ad6b admin admin] PATH_INFO: `/v3/projects/b34af39c52534239b864423d6236bbba/users/07e6e939c82c4864806c0d4df4e93a86/roles/d009411496b8452b88e77ffb8db015a7` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:42.529898 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 851/1715] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 01:22:42 2022] POST /identity/v3/projects => generated 357 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:22:42.537640 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cb1215-2193-4b4e-8368-5c0a953260d3 admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:22:42.573277 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6fbb82-6219-4b1a-8d9f-acd1e474ad6b admin admin] RBAC: Authorizing `identity:create_grant(project_id=b34af39c52534239b864423d6236bbba, user_id=07e6e939c82c4864806c0d4df4e93a86, role_id=d009411496b8452b88e77ffb8db015a7)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:22:42.576004 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6fbb82-6219-4b1a-8d9f-acd1e474ad6b admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:22:42.665497 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 865/1716] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 01:22:42 2022] PUT /identity/v3/projects/b34af39c52534239b864423d6236bbba/users/07e6e939c82c4864806c0d4df4e93a86/roles/d009411496b8452b88e77ffb8db015a7 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:22:42.666638 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6fbb82-6219-4b1a-8d9f-acd1e474ad6b admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:22:42.783262 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cb1215-2193-4b4e-8368-5c0a953260d3 admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:22:42.798706 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:22:42.875322 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6fbb82-6219-4b1a-8d9f-acd1e474ad6b admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:22:42.894299 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c499e32e-348e-4284-998b-9d87e61c60fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:22:42.894942 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c499e32e-348e-4284-998b-9d87e61c60fc admin admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:42.895545 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c499e32e-348e-4284-998b-9d87e61c60fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:42.895545 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c499e32e-348e-4284-998b-9d87e61c60fc admin admin] PATH_INFO: `/v3/roles` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:42.896763 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c499e32e-348e-4284-998b-9d87e61c60fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:22:42.898415 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c499e32e-348e-4284-998b-9d87e61c60fc admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:22:42.937936 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 866/1717] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:22:42 2022] GET /identity/v3/roles => generated 1420 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:22:42.943600 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c499e32e-348e-4284-998b-9d87e61c60fc admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:22:42.969906 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f383c6a0-d487-4eed-8bd7-a6bb94ac0120 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:22:42.970543 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f383c6a0-d487-4eed-8bd7-a6bb94ac0120 admin admin] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:42.970973 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f383c6a0-d487-4eed-8bd7-a6bb94ac0120 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:42.971112 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f383c6a0-d487-4eed-8bd7-a6bb94ac0120 admin admin] PATH_INFO: `/v3/users` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:42.971793 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f383c6a0-d487-4eed-8bd7-a6bb94ac0120 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:22:42.974378 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f383c6a0-d487-4eed-8bd7-a6bb94ac0120 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:22:42.978953 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c499e32e-348e-4284-998b-9d87e61c60fc admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:22:42.992957 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c76cc2-5456-4e42-a5f8-7c94d4e8f30a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:22:42.993651 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c76cc2-5456-4e42-a5f8-7c94d4e8f30a admin admin] REQUEST_METHOD: `PUT` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:42.993936 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c76cc2-5456-4e42-a5f8-7c94d4e8f30a admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:42.994212 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c76cc2-5456-4e42-a5f8-7c94d4e8f30a admin admin] PATH_INFO: `/v3/projects/b34af39c52534239b864423d6236bbba/users/07e6e939c82c4864806c0d4df4e93a86/roles/d009411496b8452b88e77ffb8db015a7` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:42.997345 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c76cc2-5456-4e42-a5f8-7c94d4e8f30a admin admin] RBAC: Authorizing `identity:create_grant(project_id=b34af39c52534239b864423d6236bbba, user_id=07e6e939c82c4864806c0d4df4e93a86, role_id=d009411496b8452b88e77ffb8db015a7)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:22:43.000116 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c76cc2-5456-4e42-a5f8-7c94d4e8f30a admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:22:43.024434 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 867/1718] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 01:22:42 2022] PUT /identity/v3/projects/b34af39c52534239b864423d6236bbba/users/07e6e939c82c4864806c0d4df4e93a86/roles/d009411496b8452b88e77ffb8db015a7 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:22:43.043208 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff645f47-708c-4e26-8cf8-168c1e47a040 None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:43.049807 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff645f47-708c-4e26-8cf8-168c1e47a040 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:43.049807 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff645f47-708c-4e26-8cf8-168c1e47a040 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:43.103854 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-ff645f47-708c-4e26-8cf8-168c1e47a040 None None] MFA Rules not processed for user `07e6e939c82c4864806c0d4df4e93a86`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:22:43.105835 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-ff645f47-708c-4e26-8cf8-168c1e47a040 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:22:43.131675 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 852/1719] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 01:22:42 2022] POST /identity/v3/users => generated 279 bytes in 596 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:22:43.136119 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f383c6a0-d487-4eed-8bd7-a6bb94ac0120 admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:22:43.165206 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:22:43.172458 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f383c6a0-d487-4eed-8bd7-a6bb94ac0120 admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:22:43.297863 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c57bae2-e260-457c-8501-5b9c7bfb0c4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:22:43.298523 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c57bae2-e260-457c-8501-5b9c7bfb0c4b admin admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:43.299121 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c57bae2-e260-457c-8501-5b9c7bfb0c4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:43.299441 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c57bae2-e260-457c-8501-5b9c7bfb0c4b admin admin] PATH_INFO: `/v3/roles` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:43.300499 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c57bae2-e260-457c-8501-5b9c7bfb0c4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:22:43.305230 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c57bae2-e260-457c-8501-5b9c7bfb0c4b admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:22:43.343925 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 853/1720] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:22:43 2022] GET /identity/v3/roles => generated 1420 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:22:43.343925 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0483ecc-03ab-42cc-86e5-8afe06fc4e94 None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:43.343925 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0483ecc-03ab-42cc-86e5-8afe06fc4e94 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:43.343925 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0483ecc-03ab-42cc-86e5-8afe06fc4e94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:43.378955 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 868/1721] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:22:43 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 335 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:22:43.378955 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a19276-5c92-435b-90ce-27379101603f None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:43.379204 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a19276-5c92-435b-90ce-27379101603f None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:43.379499 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a19276-5c92-435b-90ce-27379101603f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:43.436442 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-f0483ecc-03ab-42cc-86e5-8afe06fc4e94 None None] MFA Rules not processed for user `71d0408df86f4be0ac59cc80575a14e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:22:43.440474 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-f0483ecc-03ab-42cc-86e5-8afe06fc4e94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:22:43.487732 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-b8a19276-5c92-435b-90ce-27379101603f None None] MFA Rules not processed for user `63cc85f4861948e79c90753f4ce2b4e9`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:22:43.496858 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-b8a19276-5c92-435b-90ce-27379101603f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:22:43.666977 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 869/1722] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:22:43 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:22:43.668362 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a19276-5c92-435b-90ce-27379101603f None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:22:43.682574 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 854/1723] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:22:43 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 338 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:22:43.684824 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0483ecc-03ab-42cc-86e5-8afe06fc4e94 None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:22:43.691368 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-f0483ecc-03ab-42cc-86e5-8afe06fc4e94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:22:43.763609 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a19276-5c92-435b-90ce-27379101603f None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:22:43.795914 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3b09bd-ba4e-40d5-8470-572ca1511dc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:22:43.795914 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3b09bd-ba4e-40d5-8470-572ca1511dc0 admin admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:43.795914 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3b09bd-ba4e-40d5-8470-572ca1511dc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:43.795914 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3b09bd-ba4e-40d5-8470-572ca1511dc0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:43.796542 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3b09bd-ba4e-40d5-8470-572ca1511dc0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:22:43.807434 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3b09bd-ba4e-40d5-8470-572ca1511dc0 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:22:43.823277 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:22:43.856308 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 870/1724] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:22:43 2022] GET /identity/v3/roles => generated 1420 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:22:43.860266 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3b09bd-ba4e-40d5-8470-572ca1511dc0 admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:22:43.868314 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-bc3b09bd-ba4e-40d5-8470-572ca1511dc0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:22:43.883136 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0483ecc-03ab-42cc-86e5-8afe06fc4e94 None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:22:43.914098 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9f1ba7-e8d4-4446-978c-0308209d01bc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5c8fc69c518412ba1aad61a9f4f558c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:22:43.917110 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9f1ba7-e8d4-4446-978c-0308209d01bc service cinder] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:43.925503 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9f1ba7-e8d4-4446-978c-0308209d01bc service cinder] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:43.925503 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9f1ba7-e8d4-4446-978c-0308209d01bc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:43.926142 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3b09bd-ba4e-40d5-8470-572ca1511dc0 admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:22:43.926142 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d765f0a-c92f-4aad-8018-e220db754898 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:22:43.927295 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d765f0a-c92f-4aad-8018-e220db754898 admin admin] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:43.927736 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d765f0a-c92f-4aad-8018-e220db754898 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:43.928202 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d765f0a-c92f-4aad-8018-e220db754898 admin admin] PATH_INFO: `/v3/projects` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:43.929087 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d765f0a-c92f-4aad-8018-e220db754898 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:22:43.931710 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d765f0a-c92f-4aad-8018-e220db754898 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:22:43.951936 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9f1ba7-e8d4-4446-978c-0308209d01bc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:22:43.953478 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9f1ba7-e8d4-4446-978c-0308209d01bc service cinder] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:22:43.968448 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 855/1725] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:22:43 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 285 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:22:43.969386 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9f1ba7-e8d4-4446-978c-0308209d01bc service cinder] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:22:43.981756 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9f1ba7-e8d4-4446-978c-0308209d01bc service cinder] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:22:43.988346 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3924f0-c120-4ed6-b57b-7a7229f1e8bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:22:43.989754 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3924f0-c120-4ed6-b57b-7a7229f1e8bf admin admin] REQUEST_METHOD: `PUT` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:43.990146 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3924f0-c120-4ed6-b57b-7a7229f1e8bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:43.990466 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3924f0-c120-4ed6-b57b-7a7229f1e8bf admin admin] PATH_INFO: `/v3/projects/f1bb404c17954bbfa4aa9d4d14ddea0f/users/4f0a1c35f10f4e509b386358d28ef427/roles/d0b9768f4d014766b9326387f2e11f34` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:44.054099 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3924f0-c120-4ed6-b57b-7a7229f1e8bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1bb404c17954bbfa4aa9d4d14ddea0f, user_id=4f0a1c35f10f4e509b386358d28ef427, role_id=d0b9768f4d014766b9326387f2e11f34)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:22:44.059025 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3924f0-c120-4ed6-b57b-7a7229f1e8bf admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:22:44.116846 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 871/1726] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 01:22:43 2022] POST /identity/v3/projects => generated 405 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:22:44.125336 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d765f0a-c92f-4aad-8018-e220db754898 admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:22:44.171706 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 856/1727] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 01:22:43 2022] PUT /identity/v3/projects/f1bb404c17954bbfa4aa9d4d14ddea0f/users/4f0a1c35f10f4e509b386358d28ef427/roles/d0b9768f4d014766b9326387f2e11f34 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:22:44.184401 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb9113a-30e8-446b-958e-702df733c451 None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:44.185523 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb9113a-30e8-446b-958e-702df733c451 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:44.186624 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb9113a-30e8-446b-958e-702df733c451 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:44.245270 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-3bb9113a-30e8-446b-958e-702df733c451 None None] MFA Rules not processed for user `4f0a1c35f10f4e509b386358d28ef427`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:22:44.246698 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-3bb9113a-30e8-446b-958e-702df733c451 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:22:44.331952 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d765f0a-c92f-4aad-8018-e220db754898 admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:22:44.352195 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:22:44.418577 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 857/1728] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:22:44 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:22:44.423686 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb9113a-30e8-446b-958e-702df733c451 None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:22:44.490808 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68ed927-6507-4c8e-974e-706a1ead5a51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:22:44.494630 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68ed927-6507-4c8e-974e-706a1ead5a51 admin admin] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:44.495218 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68ed927-6507-4c8e-974e-706a1ead5a51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:44.496793 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68ed927-6507-4c8e-974e-706a1ead5a51 admin admin] PATH_INFO: `/v3/users` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:44.499599 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68ed927-6507-4c8e-974e-706a1ead5a51 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:22:44.505707 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68ed927-6507-4c8e-974e-706a1ead5a51 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:22:44.661552 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb9113a-30e8-446b-958e-702df733c451 None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:22:44.676555 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e223d4d2-c1c6-414a-9235-4c21da3bb3d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:22:44.678047 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e223d4d2-c1c6-414a-9235-4c21da3bb3d2 admin admin] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:44.678449 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e223d4d2-c1c6-414a-9235-4c21da3bb3d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:44.678839 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e223d4d2-c1c6-414a-9235-4c21da3bb3d2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:44.680333 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e223d4d2-c1c6-414a-9235-4c21da3bb3d2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:22:44.692617 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e223d4d2-c1c6-414a-9235-4c21da3bb3d2 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:22:44.706439 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 872/1729] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 01:22:44 2022] POST /identity/v3/users => generated 309 bytes in 582 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:22:44.707446 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68ed927-6507-4c8e-974e-706a1ead5a51 admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:22:44.711868 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-e68ed927-6507-4c8e-974e-706a1ead5a51 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:22:44.760526 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68ed927-6507-4c8e-974e-706a1ead5a51 admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:22:44.774855 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3ab875-896b-4a01-ac81-adeb0e878f0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:22:44.781821 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3ab875-896b-4a01-ac81-adeb0e878f0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:44.794622 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3ab875-896b-4a01-ac81-adeb0e878f0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:44.794622 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3ab875-896b-4a01-ac81-adeb0e878f0f admin admin] PATH_INFO: `/v3/projects/4b5fd32565144659916a7686f464afef` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:44.794622 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3ab875-896b-4a01-ac81-adeb0e878f0f admin admin] RBAC: Authorizing `identity:delete_project(project_id=4b5fd32565144659916a7686f464afef)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:22:44.794622 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3ab875-896b-4a01-ac81-adeb0e878f0f admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:22:44.853117 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 858/1730] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 01:22:44 2022] POST /identity/v3/projects => generated 357 bytes in 429 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:22:44.853117 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e223d4d2-c1c6-414a-9235-4c21da3bb3d2 admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:22:44.940869 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:22:45.042817 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e223d4d2-c1c6-414a-9235-4c21da3bb3d2 admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:22:45.042937 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-ad3ab875-896b-4a01-ac81-adeb0e878f0f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b5fd32565144659916a7686f464afef', 'request_id': 'req-ad3ab875-896b-4a01-ac81-adeb0e878f0f'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:22:45.065908 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-ad3ab875-896b-4a01-ac81-adeb0e878f0f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b5fd32565144659916a7686f464afef', 'request_id': 'req-ad3ab875-896b-4a01-ac81-adeb0e878f0f'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:22:45.157973 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-ad3ab875-896b-4a01-ac81-adeb0e878f0f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b5fd32565144659916a7686f464afef', 'request_id': 'req-ad3ab875-896b-4a01-ac81-adeb0e878f0f'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:22:45.187319 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-ad3ab875-896b-4a01-ac81-adeb0e878f0f admin admin] The token cache is being invalidate because project 4b5fd32565144659916a7686f464afef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81750) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 01:22:45.192229 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-ad3ab875-896b-4a01-ac81-adeb0e878f0f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:22:45.192229 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-ad3ab875-896b-4a01-ac81-adeb0e878f0f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:22:45.195049 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 873/1731] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 01:22:44 2022] DELETE /identity/v3/projects/4b5fd32565144659916a7686f464afef => generated 0 bytes in 488 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:22:45.196124 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3ab875-896b-4a01-ac81-adeb0e878f0f admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:22:45.206534 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-ad3ab875-896b-4a01-ac81-adeb0e878f0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:22:45.242674 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89a3952-86ff-403e-9dbf-9f8627a89738 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:22:45.245332 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89a3952-86ff-403e-9dbf-9f8627a89738 admin admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:45.247009 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89a3952-86ff-403e-9dbf-9f8627a89738 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:45.247413 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89a3952-86ff-403e-9dbf-9f8627a89738 admin admin] PATH_INFO: `/v3/roles` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:45.250804 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89a3952-86ff-403e-9dbf-9f8627a89738 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:22:45.254665 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89a3952-86ff-403e-9dbf-9f8627a89738 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:22:45.296220 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 859/1732] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:22:44 2022] GET /identity/v3/roles => generated 1420 bytes in 443 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:22:45.297576 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89a3952-86ff-403e-9dbf-9f8627a89738 admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:22:45.304835 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-d89a3952-86ff-403e-9dbf-9f8627a89738 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:22:45.359734 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89a3952-86ff-403e-9dbf-9f8627a89738 admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:22:45.373975 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d938dd-63b0-4981-b4e5-94e7d838823d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:22:45.381201 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d938dd-63b0-4981-b4e5-94e7d838823d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:45.382535 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d938dd-63b0-4981-b4e5-94e7d838823d admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:45.386113 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d938dd-63b0-4981-b4e5-94e7d838823d admin admin] PATH_INFO: `/v3/users/9e23f4ccbd244889ae9c02fd7802901c` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:45.388216 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d938dd-63b0-4981-b4e5-94e7d838823d admin admin] RBAC: Authorizing `identity:delete_user(user_id=9e23f4ccbd244889ae9c02fd7802901c)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:22:45.393419 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d938dd-63b0-4981-b4e5-94e7d838823d admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:22:45.434515 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3ab875-896b-4a01-ac81-adeb0e878f0f admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:22:45.464035 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46c46f8-5044-458f-8dff-49424f03980a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:22:45.482010 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46c46f8-5044-458f-8dff-49424f03980a admin admin] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:45.482010 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46c46f8-5044-458f-8dff-49424f03980a admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:45.482010 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46c46f8-5044-458f-8dff-49424f03980a admin admin] PATH_INFO: `/v3/users` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:45.482010 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46c46f8-5044-458f-8dff-49424f03980a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:22:45.482010 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46c46f8-5044-458f-8dff-49424f03980a admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:22:45.610050 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 874/1733] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 01:22:45 2022] POST /identity/v3/users => generated 285 bytes in 414 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:22:45.611044 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46c46f8-5044-458f-8dff-49424f03980a admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:22:45.613053 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-d46c46f8-5044-458f-8dff-49424f03980a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:22:45.636516 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46c46f8-5044-458f-8dff-49424f03980a admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:22:45.642551 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f992560c-e445-4114-9099-748eaba13114 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:22:45.644977 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f992560c-e445-4114-9099-748eaba13114 admin admin] REQUEST_METHOD: `PUT` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:45.645680 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f992560c-e445-4114-9099-748eaba13114 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:45.646408 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f992560c-e445-4114-9099-748eaba13114 admin admin] PATH_INFO: `/v3/projects/53be4396b4934ce9ac8a4e5cdd81333b/users/13c39c33f32e44c9b3647028cf0b5b19/roles/d009411496b8452b88e77ffb8db015a7` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:45.671480 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f992560c-e445-4114-9099-748eaba13114 admin admin] RBAC: Authorizing `identity:create_grant(project_id=53be4396b4934ce9ac8a4e5cdd81333b, user_id=13c39c33f32e44c9b3647028cf0b5b19, role_id=d009411496b8452b88e77ffb8db015a7)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:22:45.689804 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f992560c-e445-4114-9099-748eaba13114 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:22:45.744272 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-12d938dd-63b0-4981-b4e5-94e7d838823d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9e23f4ccbd244889ae9c02fd7802901c'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:22:45.799118 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 875/1734] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 01:22:45 2022] PUT /identity/v3/projects/53be4396b4934ce9ac8a4e5cdd81333b/users/13c39c33f32e44c9b3647028cf0b5b19/roles/d009411496b8452b88e77ffb8db015a7 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:22:45.803149 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549d0993-80fd-427b-ad88-2c0c0aa21d93 None None] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:45.803757 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549d0993-80fd-427b-ad88-2c0c0aa21d93 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:45.804295 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549d0993-80fd-427b-ad88-2c0c0aa21d93 None None] PATH_INFO: `/` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:45.805914 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 876/1735] 10.222.0.20 () {54 vars in 921 bytes} [Wed Aug 31 01:22:45 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:22:45.811795 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-12d938dd-63b0-4981-b4e5-94e7d838823d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9e23f4ccbd244889ae9c02fd7802901c'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:22:45.812482 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549d0993-80fd-427b-ad88-2c0c0aa21d93 None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:22:45.813284 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-12d938dd-63b0-4981-b4e5-94e7d838823d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9e23f4ccbd244889ae9c02fd7802901c'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:22:45.824557 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-12d938dd-63b0-4981-b4e5-94e7d838823d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9e23f4ccbd244889ae9c02fd7802901c'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:22:45.825502 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-12d938dd-63b0-4981-b4e5-94e7d838823d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9e23f4ccbd244889ae9c02fd7802901c'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:22:45.888318 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 860/1736] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 01:22:45 2022] DELETE /identity/v3/users/9e23f4ccbd244889ae9c02fd7802901c => generated 0 bytes in 581 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:22:45.891552 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d938dd-63b0-4981-b4e5-94e7d838823d admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:22:45.899733 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-12d938dd-63b0-4981-b4e5-94e7d838823d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:22:45.903185 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=86,keystone:INSERT=7,keystone:DELETE=7 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:22:46.078765 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549d0993-80fd-427b-ad88-2c0c0aa21d93 None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:22:46.086178 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d938dd-63b0-4981-b4e5-94e7d838823d admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:22:46.100454 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc9923a-f014-4cd2-a240-a4988c5700c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:22:46.101143 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc9923a-f014-4cd2-a240-a4988c5700c8 admin admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:46.101435 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc9923a-f014-4cd2-a240-a4988c5700c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:46.105158 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc9923a-f014-4cd2-a240-a4988c5700c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:46.105916 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc9923a-f014-4cd2-a240-a4988c5700c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:22:46.109651 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc9923a-f014-4cd2-a240-a4988c5700c8 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:22:46.124802 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60623128-159d-4496-859b-e7a35996cb79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:22:46.124802 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60623128-159d-4496-859b-e7a35996cb79 admin admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:46.124802 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60623128-159d-4496-859b-e7a35996cb79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:46.124802 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60623128-159d-4496-859b-e7a35996cb79 admin admin] PATH_INFO: `/v3/roles` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:46.125266 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60623128-159d-4496-859b-e7a35996cb79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:22:46.136492 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60623128-159d-4496-859b-e7a35996cb79 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:22:46.156618 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 861/1737] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:22:45 2022] GET /identity/v3/roles => generated 1420 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:22:46.160848 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05173b55-5b91-4150-ba41-4a1544718bc4 None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:46.160848 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05173b55-5b91-4150-ba41-4a1544718bc4 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:46.160848 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05173b55-5b91-4150-ba41-4a1544718bc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:46.174476 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 877/1738] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:22:45 2022] GET /identity/v3/roles => generated 1420 bytes in 368 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:22:46.175358 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60623128-159d-4496-859b-e7a35996cb79 admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:22:46.195071 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-05173b55-5b91-4150-ba41-4a1544718bc4 None None] MFA Rules not processed for user `8744c9de803048f583d0317d01f6a540`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:22:46.199510 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-05173b55-5b91-4150-ba41-4a1544718bc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:22:46.214616 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60623128-159d-4496-859b-e7a35996cb79 admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:22:46.226776 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e93074-eba4-4d4f-9ff8-53fe7dfdd0e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:22:46.234066 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e93074-eba4-4d4f-9ff8-53fe7dfdd0e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:46.234066 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e93074-eba4-4d4f-9ff8-53fe7dfdd0e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:46.234066 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e93074-eba4-4d4f-9ff8-53fe7dfdd0e4 admin admin] PATH_INFO: `/v3/projects/53be4396b4934ce9ac8a4e5cdd81333b/users/13c39c33f32e44c9b3647028cf0b5b19/roles/d009411496b8452b88e77ffb8db015a7` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:46.234066 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e93074-eba4-4d4f-9ff8-53fe7dfdd0e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=53be4396b4934ce9ac8a4e5cdd81333b, user_id=13c39c33f32e44c9b3647028cf0b5b19, role_id=d009411496b8452b88e77ffb8db015a7)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:22:46.243094 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e93074-eba4-4d4f-9ff8-53fe7dfdd0e4 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:22:46.277416 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 878/1739] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 01:22:46 2022] PUT /identity/v3/projects/53be4396b4934ce9ac8a4e5cdd81333b/users/13c39c33f32e44c9b3647028cf0b5b19/roles/d009411496b8452b88e77ffb8db015a7 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:22:46.277416 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e93074-eba4-4d4f-9ff8-53fe7dfdd0e4 admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:22:46.295971 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-02e93074-eba4-4d4f-9ff8-53fe7dfdd0e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:22:46.413820 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:22:46.478379 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e93074-eba4-4d4f-9ff8-53fe7dfdd0e4 admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:22:46.486056 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 862/1740] 10.222.0.20 () {58 vars in 1018 bytes} [Wed Aug 31 01:22:46 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 329 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:22:46.494071 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05173b55-5b91-4150-ba41-4a1544718bc4 None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:22:46.495737 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-05173b55-5b91-4150-ba41-4a1544718bc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:22:46.496831 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403b4f23-4f40-4a65-9c8f-3589b4881402 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:22:46.497512 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403b4f23-4f40-4a65-9c8f-3589b4881402 admin admin] REQUEST_METHOD: `PUT` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:46.497793 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403b4f23-4f40-4a65-9c8f-3589b4881402 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:46.498068 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403b4f23-4f40-4a65-9c8f-3589b4881402 admin admin] PATH_INFO: `/v3/projects/6d28bd8815444d80b7a82db2a8cf4be8/users/7998e2533226409cb62d5a33693757f1/roles/d009411496b8452b88e77ffb8db015a7` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:46.530989 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403b4f23-4f40-4a65-9c8f-3589b4881402 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d28bd8815444d80b7a82db2a8cf4be8, user_id=7998e2533226409cb62d5a33693757f1, role_id=d009411496b8452b88e77ffb8db015a7)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:22:46.533898 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403b4f23-4f40-4a65-9c8f-3589b4881402 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:22:46.619144 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 879/1741] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 01:22:46 2022] PUT /identity/v3/projects/6d28bd8815444d80b7a82db2a8cf4be8/users/7998e2533226409cb62d5a33693757f1/roles/d009411496b8452b88e77ffb8db015a7 => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:22:46.620125 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403b4f23-4f40-4a65-9c8f-3589b4881402 admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:22:46.629078 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05173b55-5b91-4150-ba41-4a1544718bc4 None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:22:46.716970 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60cdaa7-96dc-45a2-a8c9-a81b72110a49 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5c8fc69c518412ba1aad61a9f4f558c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:22:46.722089 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60cdaa7-96dc-45a2-a8c9-a81b72110a49 service cinder] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:46.722618 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60cdaa7-96dc-45a2-a8c9-a81b72110a49 service cinder] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:46.723125 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60cdaa7-96dc-45a2-a8c9-a81b72110a49 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:46.726673 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-c60cdaa7-96dc-45a2-a8c9-a81b72110a49 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:22:46.774400 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403b4f23-4f40-4a65-9c8f-3589b4881402 admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:22:46.791683 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8effee53-1be6-40c4-934d-0e12808bcd9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:22:46.792559 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8effee53-1be6-40c4-934d-0e12808bcd9c admin admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:46.793057 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8effee53-1be6-40c4-934d-0e12808bcd9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:46.793469 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8effee53-1be6-40c4-934d-0e12808bcd9c admin admin] PATH_INFO: `/v3/roles` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:46.794288 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8effee53-1be6-40c4-934d-0e12808bcd9c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:22:46.796244 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8effee53-1be6-40c4-934d-0e12808bcd9c admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:22:46.817376 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 880/1742] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:22:46 2022] GET /identity/v3/roles => generated 1420 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:22:46.818389 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8effee53-1be6-40c4-934d-0e12808bcd9c admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:22:46.842873 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8effee53-1be6-40c4-934d-0e12808bcd9c admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:22:46.858627 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa3f670-3642-4c7a-ab31-e6999ecd8493 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:22:46.859326 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa3f670-3642-4c7a-ab31-e6999ecd8493 admin admin] REQUEST_METHOD: `PUT` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:46.859605 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa3f670-3642-4c7a-ab31-e6999ecd8493 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:46.859881 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa3f670-3642-4c7a-ab31-e6999ecd8493 admin admin] PATH_INFO: `/v3/projects/6d28bd8815444d80b7a82db2a8cf4be8/users/7998e2533226409cb62d5a33693757f1/roles/d009411496b8452b88e77ffb8db015a7` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:46.873691 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa3f670-3642-4c7a-ab31-e6999ecd8493 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d28bd8815444d80b7a82db2a8cf4be8, user_id=7998e2533226409cb62d5a33693757f1, role_id=d009411496b8452b88e77ffb8db015a7)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:22:46.876811 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa3f670-3642-4c7a-ab31-e6999ecd8493 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:22:46.920406 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 881/1743] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 01:22:46 2022] PUT /identity/v3/projects/6d28bd8815444d80b7a82db2a8cf4be8/users/7998e2533226409cb62d5a33693757f1/roles/d009411496b8452b88e77ffb8db015a7 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:22:46.922206 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33741691-8d1d-4f14-b18b-521763c0ea1c None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:46.922559 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33741691-8d1d-4f14-b18b-521763c0ea1c None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:46.922893 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33741691-8d1d-4f14-b18b-521763c0ea1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:46.945382 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60cdaa7-96dc-45a2-a8c9-a81b72110a49 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:22:46.946965 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60cdaa7-96dc-45a2-a8c9-a81b72110a49 service cinder] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:22:46.964087 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-33741691-8d1d-4f14-b18b-521763c0ea1c None None] MFA Rules not processed for user `7998e2533226409cb62d5a33693757f1`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:22:46.967699 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-33741691-8d1d-4f14-b18b-521763c0ea1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:22:46.969724 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 863/1744] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:22:46 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 476 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 01:22:47.275108 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 882/1745] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:22:46 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 349 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:22:47.283072 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60cdaa7-96dc-45a2-a8c9-a81b72110a49 service cinder] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:22:47.284623 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-c60cdaa7-96dc-45a2-a8c9-a81b72110a49 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:22:47.324094 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60cdaa7-96dc-45a2-a8c9-a81b72110a49 service cinder] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:22:47.437633 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5df7296-9c86-4f7a-8893-6c0c8cb1d83b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5c8fc69c518412ba1aad61a9f4f558c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:22:47.438426 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5df7296-9c86-4f7a-8893-6c0c8cb1d83b service cinder] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:47.438774 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5df7296-9c86-4f7a-8893-6c0c8cb1d83b service cinder] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:47.439165 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5df7296-9c86-4f7a-8893-6c0c8cb1d83b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:47.453656 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5df7296-9c86-4f7a-8893-6c0c8cb1d83b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:22:47.455951 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5df7296-9c86-4f7a-8893-6c0c8cb1d83b service cinder] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:22:47.473326 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 864/1746] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:22:47 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:22:47.489780 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33741691-8d1d-4f14-b18b-521763c0ea1c None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:22:47.491116 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-33741691-8d1d-4f14-b18b-521763c0ea1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:22:47.524454 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:22:47.754658 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33741691-8d1d-4f14-b18b-521763c0ea1c None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:22:47.769527 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0fcced-1509-4434-b7e5-5e705ebf2a16 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8744c9de803048f583d0317d01f6a540', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:22:47.770169 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0fcced-1509-4434-b7e5-5e705ebf2a16 service nova] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:47.770452 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0fcced-1509-4434-b7e5-5e705ebf2a16 service nova] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:47.771253 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0fcced-1509-4434-b7e5-5e705ebf2a16 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:47.793969 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0fcced-1509-4434-b7e5-5e705ebf2a16 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:22:47.797296 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5df7296-9c86-4f7a-8893-6c0c8cb1d83b service cinder] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:22:47.808179 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5df7296-9c86-4f7a-8893-6c0c8cb1d83b service cinder] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:22:47.810359 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0fcced-1509-4434-b7e5-5e705ebf2a16 service nova] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:22:47.821682 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6912ade3-fe6a-4d5c-8cee-a7270d4574db service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5c8fc69c518412ba1aad61a9f4f558c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:22:47.824286 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6912ade3-fe6a-4d5c-8cee-a7270d4574db service cinder] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:47.824583 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6912ade3-fe6a-4d5c-8cee-a7270d4574db service cinder] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:47.826410 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6912ade3-fe6a-4d5c-8cee-a7270d4574db service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:47.831833 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-6912ade3-fe6a-4d5c-8cee-a7270d4574db service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:22:47.848485 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 883/1747] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:22:47 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:22:47.966646 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:22:48.014216 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6912ade3-fe6a-4d5c-8cee-a7270d4574db service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:22:48.015319 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6912ade3-fe6a-4d5c-8cee-a7270d4574db service cinder] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:22:48.064085 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 865/1748] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:22:47 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 267 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 01:22:48.610844 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0fcced-1509-4434-b7e5-5e705ebf2a16 service nova] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:22:48.612447 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-ff0fcced-1509-4434-b7e5-5e705ebf2a16 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:22:48.858117 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:22:48.921675 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0fcced-1509-4434-b7e5-5e705ebf2a16 service nova] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:22:48.946034 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d934e72-4e62-4e48-9ba8-a7e9bcaac5a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:22:48.946711 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d934e72-4e62-4e48-9ba8-a7e9bcaac5a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:48.947046 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d934e72-4e62-4e48-9ba8-a7e9bcaac5a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:48.948064 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d934e72-4e62-4e48-9ba8-a7e9bcaac5a7 admin admin] PATH_INFO: `/v3/projects/2ab36940693d4ab185eb56876959d34f` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:48.961767 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d934e72-4e62-4e48-9ba8-a7e9bcaac5a7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ab36940693d4ab185eb56876959d34f)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:22:48.966889 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d934e72-4e62-4e48-9ba8-a7e9bcaac5a7 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:22:49.178907 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-2d934e72-4e62-4e48-9ba8-a7e9bcaac5a7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ab36940693d4ab185eb56876959d34f', 'request_id': 'req-2d934e72-4e62-4e48-9ba8-a7e9bcaac5a7'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:22:49.187840 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-2d934e72-4e62-4e48-9ba8-a7e9bcaac5a7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ab36940693d4ab185eb56876959d34f', 'request_id': 'req-2d934e72-4e62-4e48-9ba8-a7e9bcaac5a7'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:22:49.304755 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-2d934e72-4e62-4e48-9ba8-a7e9bcaac5a7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ab36940693d4ab185eb56876959d34f', 'request_id': 'req-2d934e72-4e62-4e48-9ba8-a7e9bcaac5a7'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:22:49.357501 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-2d934e72-4e62-4e48-9ba8-a7e9bcaac5a7 admin admin] The token cache is being invalidate because project 2ab36940693d4ab185eb56876959d34f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81750) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 01:22:49.357501 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-2d934e72-4e62-4e48-9ba8-a7e9bcaac5a7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:22:49.357501 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-2d934e72-4e62-4e48-9ba8-a7e9bcaac5a7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:22:49.361198 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 884/1749] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 01:22:48 2022] DELETE /identity/v3/projects/2ab36940693d4ab185eb56876959d34f => generated 0 bytes in 751 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:22:49.366534 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee616cd-d16c-4ff4-9ae6-645bf7d60aa3 None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:49.367044 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee616cd-d16c-4ff4-9ae6-645bf7d60aa3 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:49.367231 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee616cd-d16c-4ff4-9ae6-645bf7d60aa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:49.433434 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-7ee616cd-d16c-4ff4-9ae6-645bf7d60aa3 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:22:49.440588 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-7ee616cd-d16c-4ff4-9ae6-645bf7d60aa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:22:49.730616 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 866/1750] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:22:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 361 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:22:49.736216 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2eab9f5-d717-4e95-91ae-e8837d20d80c None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:49.736757 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2eab9f5-d717-4e95-91ae-e8837d20d80c None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:49.736757 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2eab9f5-d717-4e95-91ae-e8837d20d80c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:49.787540 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-e2eab9f5-d717-4e95-91ae-e8837d20d80c None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:22:49.794874 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-e2eab9f5-d717-4e95-91ae-e8837d20d80c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:22:49.828181 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 885/1751] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:22:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:22:49.832188 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee616cd-d16c-4ff4-9ae6-645bf7d60aa3 None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:22:49.889253 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee616cd-d16c-4ff4-9ae6-645bf7d60aa3 None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:22:49.924804 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8665b3bf-65d9-4cb7-b470-a1e58a7143cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:22:49.925927 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8665b3bf-65d9-4cb7-b470-a1e58a7143cb admin admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:49.928016 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8665b3bf-65d9-4cb7-b470-a1e58a7143cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:49.928490 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8665b3bf-65d9-4cb7-b470-a1e58a7143cb admin admin] PATH_INFO: `/v3/domains` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:49.933938 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8665b3bf-65d9-4cb7-b470-a1e58a7143cb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:22:49.933938 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8665b3bf-65d9-4cb7-b470-a1e58a7143cb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:22:49.952034 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8665b3bf-65d9-4cb7-b470-a1e58a7143cb admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:22:50.022429 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 867/1752] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Aug 31 01:22:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:22:50.026563 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2eab9f5-d717-4e95-91ae-e8837d20d80c None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:22:50.040828 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2eab9f5-d717-4e95-91ae-e8837d20d80c None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:22:50.049301 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb25987-7407-4606-b33c-546ec9784305 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:22:50.049932 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb25987-7407-4606-b33c-546ec9784305 admin admin] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:50.056249 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb25987-7407-4606-b33c-546ec9784305 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:50.056249 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb25987-7407-4606-b33c-546ec9784305 admin admin] PATH_INFO: `/v3/projects` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:50.056249 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb25987-7407-4606-b33c-546ec9784305 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:22:50.056249 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb25987-7407-4606-b33c-546ec9784305 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:22:50.233101 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 886/1753] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 01:22:50 2022] POST /identity/v3/projects => generated 387 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:22:50.236081 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8665b3bf-65d9-4cb7-b470-a1e58a7143cb admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:22:50.267027 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8665b3bf-65d9-4cb7-b470-a1e58a7143cb admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:22:50.322204 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:22:50.420942 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2a65b5-73be-4d42-86f3-89e31f8cd140 None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:50.420942 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2a65b5-73be-4d42-86f3-89e31f8cd140 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:50.420942 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2a65b5-73be-4d42-86f3-89e31f8cd140 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:50.481442 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-dc2a65b5-73be-4d42-86f3-89e31f8cd140 None None] MFA Rules not processed for user `52ed99737fd240789bb682fd9bf11be3`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:22:50.497753 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-dc2a65b5-73be-4d42-86f3-89e31f8cd140 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:22:50.504027 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb54b160-670d-4bb6-9085-073bbda18af9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:22:50.507692 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb54b160-670d-4bb6-9085-073bbda18af9 admin admin] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:50.507989 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb54b160-670d-4bb6-9085-073bbda18af9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:50.508296 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb54b160-670d-4bb6-9085-073bbda18af9 admin admin] PATH_INFO: `/v3/users` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:50.509076 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb54b160-670d-4bb6-9085-073bbda18af9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:22:50.511859 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb54b160-670d-4bb6-9085-073bbda18af9 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:22:50.657377 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 868/1754] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 01:22:50 2022] POST /identity/v3/users => generated 294 bytes in 423 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:22:50.664019 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb54b160-670d-4bb6-9085-073bbda18af9 admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:22:50.703461 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb54b160-670d-4bb6-9085-073bbda18af9 admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:22:50.715644 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e241c5-4a36-4699-8a42-009f56216daf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:22:50.720749 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e241c5-4a36-4699-8a42-009f56216daf admin admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:50.721477 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e241c5-4a36-4699-8a42-009f56216daf admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:50.721761 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e241c5-4a36-4699-8a42-009f56216daf admin admin] PATH_INFO: `/v3/roles` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:50.724391 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e241c5-4a36-4699-8a42-009f56216daf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:22:50.730079 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e241c5-4a36-4699-8a42-009f56216daf admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:22:50.764226 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 869/1755] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:22:50 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 01:22:50.765037 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e241c5-4a36-4699-8a42-009f56216daf admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:22:50.778247 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 887/1756] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:22:50 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 365 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:22:50.821884 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e241c5-4a36-4699-8a42-009f56216daf admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:22:50.865881 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68885d97-f106-4c36-bff2-c8a7f5c1ea24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:22:50.867996 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68885d97-f106-4c36-bff2-c8a7f5c1ea24 admin admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:50.870358 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68885d97-f106-4c36-bff2-c8a7f5c1ea24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:50.870804 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68885d97-f106-4c36-bff2-c8a7f5c1ea24 admin admin] PATH_INFO: `/v3/roles` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:50.871757 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68885d97-f106-4c36-bff2-c8a7f5c1ea24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:22:50.877075 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68885d97-f106-4c36-bff2-c8a7f5c1ea24 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:22:50.935060 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 870/1757] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:22:50 2022] GET /identity/v3/roles => generated 1420 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:22:50.936381 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2a65b5-73be-4d42-86f3-89e31f8cd140 None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:22:50.953990 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2a65b5-73be-4d42-86f3-89e31f8cd140 None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:22:50.963160 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8b0b12-f394-4150-a655-e7a3e584117d None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:50.963506 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8b0b12-f394-4150-a655-e7a3e584117d None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:50.963736 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8b0b12-f394-4150-a655-e7a3e584117d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:50.980911 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c3fa31-8b03-4b80-b433-10741ba95f62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:22:50.981627 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c3fa31-8b03-4b80-b433-10741ba95f62 admin admin] REQUEST_METHOD: `PUT` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:50.981911 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c3fa31-8b03-4b80-b433-10741ba95f62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:50.982425 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c3fa31-8b03-4b80-b433-10741ba95f62 admin admin] PATH_INFO: `/v3/projects/ed4df7412bbf4201b4ba63c416288240/users/de7f905daca14609a94bcce4872904af/roles/d0b9768f4d014766b9326387f2e11f34` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:51.038964 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c3fa31-8b03-4b80-b433-10741ba95f62 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed4df7412bbf4201b4ba63c416288240, user_id=de7f905daca14609a94bcce4872904af, role_id=d0b9768f4d014766b9326387f2e11f34)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:22:51.042336 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c3fa31-8b03-4b80-b433-10741ba95f62 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:22:51.126393 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-0e8b0b12-f394-4150-a655-e7a3e584117d None None] MFA Rules not processed for user `13c39c33f32e44c9b3647028cf0b5b19`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:22:51.127136 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 888/1758] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 01:22:50 2022] PUT /identity/v3/projects/ed4df7412bbf4201b4ba63c416288240/users/de7f905daca14609a94bcce4872904af/roles/d0b9768f4d014766b9326387f2e11f34 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:22:51.134042 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-0e8b0b12-f394-4150-a655-e7a3e584117d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:22:51.145059 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef77ecd-6364-4976-add1-b6aff25ea3c0 None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:51.145059 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef77ecd-6364-4976-add1-b6aff25ea3c0 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:51.145059 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef77ecd-6364-4976-add1-b6aff25ea3c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:51.276810 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-4ef77ecd-6364-4976-add1-b6aff25ea3c0 None None] MFA Rules not processed for user `de7f905daca14609a94bcce4872904af`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:22:51.276810 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-4ef77ecd-6364-4976-add1-b6aff25ea3c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:22:51.372005 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:22:51.446752 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 871/1759] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:22:50 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 478 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 01:22:51.539098 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 889/1760] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:22:51 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 401 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:22:51.546564 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8b0b12-f394-4150-a655-e7a3e584117d None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:22:51.848469 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8b0b12-f394-4150-a655-e7a3e584117d None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:22:51.856223 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544afaf4-8400-492d-acd4-733648a811de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:22:51.857163 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544afaf4-8400-492d-acd4-733648a811de admin admin] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:51.857513 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544afaf4-8400-492d-acd4-733648a811de admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:51.857893 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544afaf4-8400-492d-acd4-733648a811de admin admin] PATH_INFO: `/v3/projects` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:51.858766 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544afaf4-8400-492d-acd4-733648a811de admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:22:51.862539 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544afaf4-8400-492d-acd4-733648a811de admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:22:51.922421 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:22:52.041721 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 872/1761] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 01:22:51 2022] POST /identity/v3/projects => generated 387 bytes in 495 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:22:52.045485 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef77ecd-6364-4976-add1-b6aff25ea3c0 None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:22:52.066507 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef77ecd-6364-4976-add1-b6aff25ea3c0 None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:22:52.331802 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f507dc48-d4ab-416a-a623-dba886d9cf01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:22:52.332513 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f507dc48-d4ab-416a-a623-dba886d9cf01 admin admin] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:52.332850 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f507dc48-d4ab-416a-a623-dba886d9cf01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:52.333171 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f507dc48-d4ab-416a-a623-dba886d9cf01 admin admin] PATH_INFO: `/v3/users` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:52.333846 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f507dc48-d4ab-416a-a623-dba886d9cf01 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:22:52.342476 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f507dc48-d4ab-416a-a623-dba886d9cf01 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:22:52.497492 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 890/1762] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 01:22:52 2022] POST /identity/v3/users => generated 300 bytes in 453 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:22:52.501623 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544afaf4-8400-492d-acd4-733648a811de admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:22:52.521424 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544afaf4-8400-492d-acd4-733648a811de admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:22:52.524862 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df0d236-2b71-4365-921a-4122fbe6c789 None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:52.525227 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df0d236-2b71-4365-921a-4122fbe6c789 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:52.525609 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df0d236-2b71-4365-921a-4122fbe6c789 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:52.534329 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a406402d-2b51-4ebb-8e5c-b761be80f4a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:22:52.540013 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a406402d-2b51-4ebb-8e5c-b761be80f4a6 admin admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:52.540013 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a406402d-2b51-4ebb-8e5c-b761be80f4a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:52.540013 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a406402d-2b51-4ebb-8e5c-b761be80f4a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:52.540013 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a406402d-2b51-4ebb-8e5c-b761be80f4a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:22:52.540013 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a406402d-2b51-4ebb-8e5c-b761be80f4a6 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:22:52.576933 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-3df0d236-2b71-4365-921a-4122fbe6c789 None None] MFA Rules not processed for user `efaf57b6482149ca86a9bba26d7654f2`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:22:52.584338 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-3df0d236-2b71-4365-921a-4122fbe6c789 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:22:52.614861 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 873/1763] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:22:52 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:22:52.617762 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a406402d-2b51-4ebb-8e5c-b761be80f4a6 admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:22:52.648848 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a406402d-2b51-4ebb-8e5c-b761be80f4a6 admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:22:52.656033 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344a6af4-b781-46dc-8414-c6bac65a394a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:22:52.656876 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344a6af4-b781-46dc-8414-c6bac65a394a admin admin] REQUEST_METHOD: `PUT` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:52.657638 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344a6af4-b781-46dc-8414-c6bac65a394a admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:52.657638 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344a6af4-b781-46dc-8414-c6bac65a394a admin admin] PATH_INFO: `/v3/projects/c3696d906fb44760800af90cffe3abfa/users/1c5c9bc2860a47568ab95d5b460137b6/roles/d009411496b8452b88e77ffb8db015a7` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:52.692796 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344a6af4-b781-46dc-8414-c6bac65a394a admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3696d906fb44760800af90cffe3abfa, user_id=1c5c9bc2860a47568ab95d5b460137b6, role_id=d009411496b8452b88e77ffb8db015a7)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:22:52.695770 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344a6af4-b781-46dc-8414-c6bac65a394a admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:22:52.748886 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 891/1764] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:22:52 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:22:52.846878 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 874/1765] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 01:22:52 2022] PUT /identity/v3/projects/c3696d906fb44760800af90cffe3abfa/users/1c5c9bc2860a47568ab95d5b460137b6/roles/d009411496b8452b88e77ffb8db015a7 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:22:52.851969 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df0d236-2b71-4365-921a-4122fbe6c789 None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:22:52.865012 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df0d236-2b71-4365-921a-4122fbe6c789 None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:22:53.001644 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91546c8-e07d-448e-9165-7a901f1aacdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:22:53.002440 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91546c8-e07d-448e-9165-7a901f1aacdd admin admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:53.002877 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91546c8-e07d-448e-9165-7a901f1aacdd admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:53.003333 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91546c8-e07d-448e-9165-7a901f1aacdd admin admin] PATH_INFO: `/v3/roles` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:53.006162 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91546c8-e07d-448e-9165-7a901f1aacdd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:22:53.010772 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91546c8-e07d-448e-9165-7a901f1aacdd admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:22:53.055888 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 892/1766] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:22:52 2022] GET /identity/v3/roles => generated 1420 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:22:53.059911 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344a6af4-b781-46dc-8414-c6bac65a394a admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:22:53.075837 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344a6af4-b781-46dc-8414-c6bac65a394a admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:22:53.082790 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04479f3-d722-4273-b186-b3635fe9b140 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:22:53.084593 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04479f3-d722-4273-b186-b3635fe9b140 admin admin] REQUEST_METHOD: `PUT` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:53.085205 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04479f3-d722-4273-b186-b3635fe9b140 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:53.085733 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04479f3-d722-4273-b186-b3635fe9b140 admin admin] PATH_INFO: `/v3/projects/c3696d906fb44760800af90cffe3abfa/users/1c5c9bc2860a47568ab95d5b460137b6/roles/d009411496b8452b88e77ffb8db015a7` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:53.114948 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04479f3-d722-4273-b186-b3635fe9b140 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3696d906fb44760800af90cffe3abfa, user_id=1c5c9bc2860a47568ab95d5b460137b6, role_id=d009411496b8452b88e77ffb8db015a7)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:22:53.125642 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04479f3-d722-4273-b186-b3635fe9b140 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:22:53.171404 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 875/1767] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 01:22:53 2022] PUT /identity/v3/projects/c3696d906fb44760800af90cffe3abfa/users/1c5c9bc2860a47568ab95d5b460137b6/roles/d009411496b8452b88e77ffb8db015a7 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:22:53.182702 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf346e1-9742-4eae-a9a9-3b088ae0fede None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:53.182976 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf346e1-9742-4eae-a9a9-3b088ae0fede None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:53.183253 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf346e1-9742-4eae-a9a9-3b088ae0fede None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:53.228408 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:22:53.267458 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-acf346e1-9742-4eae-a9a9-3b088ae0fede None None] MFA Rules not processed for user `1c5c9bc2860a47568ab95d5b460137b6`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:22:53.268192 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-acf346e1-9742-4eae-a9a9-3b088ae0fede None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:22:53.474791 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 893/1768] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:22:53 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 286 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:22:53.516533 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04479f3-d722-4273-b186-b3635fe9b140 admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:22:53.518301 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-a04479f3-d722-4273-b186-b3635fe9b140 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:22:53.645994 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04479f3-d722-4273-b186-b3635fe9b140 admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:22:53.655865 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3051fdf3-5b4d-41b5-92ca-339bfc88bfb7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5c8fc69c518412ba1aad61a9f4f558c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:22:53.657646 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3051fdf3-5b4d-41b5-92ca-339bfc88bfb7 service cinder] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:53.658212 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3051fdf3-5b4d-41b5-92ca-339bfc88bfb7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:53.658924 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3051fdf3-5b4d-41b5-92ca-339bfc88bfb7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:53.684531 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3051fdf3-5b4d-41b5-92ca-339bfc88bfb7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:22:53.693374 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3051fdf3-5b4d-41b5-92ca-339bfc88bfb7 service cinder] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:22:53.705399 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 876/1769] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:22:53 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:22:53.852560 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf346e1-9742-4eae-a9a9-3b088ae0fede None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:22:53.928454 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3051fdf3-5b4d-41b5-92ca-339bfc88bfb7 service cinder] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:22:53.938187 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3051fdf3-5b4d-41b5-92ca-339bfc88bfb7 service cinder] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:22:53.956102 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff74f320-7670-4276-b0b8-4ac2c0acc356 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5c8fc69c518412ba1aad61a9f4f558c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:22:53.956662 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff74f320-7670-4276-b0b8-4ac2c0acc356 service cinder] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:53.956976 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff74f320-7670-4276-b0b8-4ac2c0acc356 service cinder] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:53.961230 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff74f320-7670-4276-b0b8-4ac2c0acc356 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:53.966912 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-ff74f320-7670-4276-b0b8-4ac2c0acc356 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:22:54.013267 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf346e1-9742-4eae-a9a9-3b088ae0fede None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:22:54.028422 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34b62db-75b1-4970-8f1c-8c69a236ed04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:22:54.029883 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34b62db-75b1-4970-8f1c-8c69a236ed04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:54.030347 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34b62db-75b1-4970-8f1c-8c69a236ed04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:54.030822 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34b62db-75b1-4970-8f1c-8c69a236ed04 admin admin] PATH_INFO: `/v3/users/de7f905daca14609a94bcce4872904af` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:54.040260 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34b62db-75b1-4970-8f1c-8c69a236ed04 admin admin] RBAC: Authorizing `identity:delete_user(user_id=de7f905daca14609a94bcce4872904af)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:22:54.054425 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34b62db-75b1-4970-8f1c-8c69a236ed04 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:22:54.193781 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff74f320-7670-4276-b0b8-4ac2c0acc356 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:22:54.202722 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff74f320-7670-4276-b0b8-4ac2c0acc356 service cinder] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:22:54.280067 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 877/1770] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:22:53 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 01:22:54.307008 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-c34b62db-75b1-4970-8f1c-8c69a236ed04 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de7f905daca14609a94bcce4872904af'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:22:54.320124 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-c34b62db-75b1-4970-8f1c-8c69a236ed04 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de7f905daca14609a94bcce4872904af'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:22:54.320928 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-c34b62db-75b1-4970-8f1c-8c69a236ed04 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de7f905daca14609a94bcce4872904af'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:22:54.327984 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-c34b62db-75b1-4970-8f1c-8c69a236ed04 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de7f905daca14609a94bcce4872904af'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:22:54.430946 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-c34b62db-75b1-4970-8f1c-8c69a236ed04 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de7f905daca14609a94bcce4872904af'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:22:54.437829 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 894/1771] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 01:22:53 2022] DELETE /identity/v3/users/de7f905daca14609a94bcce4872904af => generated 0 bytes in 587 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:22:54.445814 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d99c91-29d6-48f2-9bae-2bf4f34f7f7f None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:54.445814 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d99c91-29d6-48f2-9bae-2bf4f34f7f7f None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:54.446083 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d99c91-29d6-48f2-9bae-2bf4f34f7f7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:54.490363 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-a0d99c91-29d6-48f2-9bae-2bf4f34f7f7f None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:22:54.493260 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-a0d99c91-29d6-48f2-9bae-2bf4f34f7f7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:22:54.657124 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:22:54.689851 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:22:54.725471 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 878/1772] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:22:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:22:54.740175 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a037fb81-2dcb-491e-ad30-7288ff9a0cc0 None None] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:54.740175 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a037fb81-2dcb-491e-ad30-7288ff9a0cc0 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:54.740175 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a037fb81-2dcb-491e-ad30-7288ff9a0cc0 None None] PATH_INFO: `/` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:54.740175 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 895/1773] 10.222.0.20 () {54 vars in 921 bytes} [Wed Aug 31 01:22:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:22:54.775179 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77880c4-36e9-4f8e-919a-eff809092849 None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:54.775473 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77880c4-36e9-4f8e-919a-eff809092849 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:54.775761 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77880c4-36e9-4f8e-919a-eff809092849 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:54.808915 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a037fb81-2dcb-491e-ad30-7288ff9a0cc0 None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:22:54.811243 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-a037fb81-2dcb-491e-ad30-7288ff9a0cc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:22:54.841357 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-b77880c4-36e9-4f8e-919a-eff809092849 None None] MFA Rules not processed for user `8744c9de803048f583d0317d01f6a540`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:22:54.843105 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-b77880c4-36e9-4f8e-919a-eff809092849 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:22:54.936854 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a037fb81-2dcb-491e-ad30-7288ff9a0cc0 None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:22:54.973414 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2a1065-df74-4954-8971-2904c5d84f1c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0817c3320d5d41d8bba0550c8ae8dd60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:22:54.973414 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2a1065-df74-4954-8971-2904c5d84f1c service neutron] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:54.975509 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2a1065-df74-4954-8971-2904c5d84f1c service neutron] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:54.976069 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2a1065-df74-4954-8971-2904c5d84f1c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:55.032829 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2a1065-df74-4954-8971-2904c5d84f1c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:22:55.040430 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2a1065-df74-4954-8971-2904c5d84f1c service neutron] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:22:55.063355 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 879/1774] 10.222.0.20 () {58 vars in 1018 bytes} [Wed Aug 31 01:22:54 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 290 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:22:55.077205 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 896/1775] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:22:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 269 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:22:55.561824 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e793c13-ac08-4bde-a179-3326d0263cb4 None None] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:55.562426 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e793c13-ac08-4bde-a179-3326d0263cb4 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:55.562856 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e793c13-ac08-4bde-a179-3326d0263cb4 None None] PATH_INFO: `/` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:55.564391 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 880/1776] 10.222.0.20 () {54 vars in 921 bytes} [Wed Aug 31 01:22:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:22:55.664805 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4a8577-240c-4381-90ae-c608c827d682 None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:55.664805 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4a8577-240c-4381-90ae-c608c827d682 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:55.664805 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4a8577-240c-4381-90ae-c608c827d682 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:55.748231 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-0b4a8577-240c-4381-90ae-c608c827d682 None None] MFA Rules not processed for user `dea0fe5b1d6b405cb47b95192580303b`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:22:55.748231 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-0b4a8577-240c-4381-90ae-c608c827d682 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:22:55.842339 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 897/1777] 10.222.0.20 () {58 vars in 1018 bytes} [Wed Aug 31 01:22:55 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:22:55.875989 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468a2b79-a14f-42a7-9192-2b0b438ef459 None None] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:55.877461 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468a2b79-a14f-42a7-9192-2b0b438ef459 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:55.877933 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468a2b79-a14f-42a7-9192-2b0b438ef459 None None] PATH_INFO: `/` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:55.881728 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 881/1778] 10.222.0.20 () {54 vars in 961 bytes} [Wed Aug 31 01:22:55 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:22:55.890937 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a914fe04-edc8-4aec-96ee-3c05ece3add3 None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:55.890937 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a914fe04-edc8-4aec-96ee-3c05ece3add3 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:55.890937 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a914fe04-edc8-4aec-96ee-3c05ece3add3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:55.924831 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-a914fe04-edc8-4aec-96ee-3c05ece3add3 None None] MFA Rules not processed for user `dea0fe5b1d6b405cb47b95192580303b`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:22:55.927089 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-a914fe04-edc8-4aec-96ee-3c05ece3add3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:22:55.937057 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 898/1779] 10.222.0.20 () {58 vars in 1058 bytes} [Wed Aug 31 01:22:55 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:22:55.953777 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468a2b79-a14f-42a7-9192-2b0b438ef459 None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:22:55.986600 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468a2b79-a14f-42a7-9192-2b0b438ef459 None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:22:56.009872 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf9be6a-6518-4810-8c4a-4ab664dff8b8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dea0fe5b1d6b405cb47b95192580303b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:22:56.010983 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf9be6a-6518-4810-8c4a-4ab664dff8b8 service placement] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:56.011320 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf9be6a-6518-4810-8c4a-4ab664dff8b8 service placement] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:56.011612 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf9be6a-6518-4810-8c4a-4ab664dff8b8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:56.021152 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf9be6a-6518-4810-8c4a-4ab664dff8b8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:22:56.023441 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf9be6a-6518-4810-8c4a-4ab664dff8b8 service placement] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:22:56.047844 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 882/1780] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:22:55 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:22:56.202961 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a914fe04-edc8-4aec-96ee-3c05ece3add3 None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:22:56.205042 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-a914fe04-edc8-4aec-96ee-3c05ece3add3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:22:56.224410 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a914fe04-edc8-4aec-96ee-3c05ece3add3 None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:22:56.228387 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0adcef-f38f-46fb-a0cb-a504b99c77cc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8744c9de803048f583d0317d01f6a540', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:22:56.229449 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0adcef-f38f-46fb-a0cb-a504b99c77cc service nova] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:56.229743 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0adcef-f38f-46fb-a0cb-a504b99c77cc service nova] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:56.230028 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0adcef-f38f-46fb-a0cb-a504b99c77cc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:56.241449 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0adcef-f38f-46fb-a0cb-a504b99c77cc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:22:56.243243 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0adcef-f38f-46fb-a0cb-a504b99c77cc service nova] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:22:56.269161 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 899/1781] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:22:56 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:22:56.533117 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf9be6a-6518-4810-8c4a-4ab664dff8b8 service placement] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:22:56.535166 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-1bf9be6a-6518-4810-8c4a-4ab664dff8b8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:22:56.598781 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf9be6a-6518-4810-8c4a-4ab664dff8b8 service placement] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:22:56.601456 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6c3d8f-8deb-48a1-8343-882e35bb9505 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:22:56.603038 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6c3d8f-8deb-48a1-8343-882e35bb9505 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:56.605612 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6c3d8f-8deb-48a1-8343-882e35bb9505 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:56.606209 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6c3d8f-8deb-48a1-8343-882e35bb9505 admin admin] PATH_INFO: `/v3/projects/ed4df7412bbf4201b4ba63c416288240` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:56.607610 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6c3d8f-8deb-48a1-8343-882e35bb9505 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ed4df7412bbf4201b4ba63c416288240)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:22:56.610408 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6c3d8f-8deb-48a1-8343-882e35bb9505 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:22:56.793128 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-5e6c3d8f-8deb-48a1-8343-882e35bb9505 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ed4df7412bbf4201b4ba63c416288240', 'request_id': 'req-5e6c3d8f-8deb-48a1-8343-882e35bb9505'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:22:56.799507 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-5e6c3d8f-8deb-48a1-8343-882e35bb9505 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ed4df7412bbf4201b4ba63c416288240', 'request_id': 'req-5e6c3d8f-8deb-48a1-8343-882e35bb9505'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:22:56.807102 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-5e6c3d8f-8deb-48a1-8343-882e35bb9505 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ed4df7412bbf4201b4ba63c416288240', 'request_id': 'req-5e6c3d8f-8deb-48a1-8343-882e35bb9505'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:22:56.918941 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-5e6c3d8f-8deb-48a1-8343-882e35bb9505 admin admin] The token cache is being invalidate because project ed4df7412bbf4201b4ba63c416288240 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81749) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 01:22:56.919818 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-5e6c3d8f-8deb-48a1-8343-882e35bb9505 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:22:56.923174 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-5e6c3d8f-8deb-48a1-8343-882e35bb9505 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:22:56.925226 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 883/1782] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 01:22:56 2022] DELETE /identity/v3/projects/ed4df7412bbf4201b4ba63c416288240 => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:22:56.930191 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0adcef-f38f-46fb-a0cb-a504b99c77cc service nova] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:22:56.955016 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0adcef-f38f-46fb-a0cb-a504b99c77cc service nova] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:22:57.019929 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62504017-8bc4-4541-99b7-a1ffb705e0f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:22:57.023027 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62504017-8bc4-4541-99b7-a1ffb705e0f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:57.023027 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62504017-8bc4-4541-99b7-a1ffb705e0f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:57.023027 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62504017-8bc4-4541-99b7-a1ffb705e0f0 admin admin] PATH_INFO: `/v3/users/1c5c9bc2860a47568ab95d5b460137b6` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:57.023027 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62504017-8bc4-4541-99b7-a1ffb705e0f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c5c9bc2860a47568ab95d5b460137b6)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:22:57.024243 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62504017-8bc4-4541-99b7-a1ffb705e0f0 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:22:57.068160 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:22:57.307309 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-62504017-8bc4-4541-99b7-a1ffb705e0f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c5c9bc2860a47568ab95d5b460137b6'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:22:57.334309 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-62504017-8bc4-4541-99b7-a1ffb705e0f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c5c9bc2860a47568ab95d5b460137b6'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:22:57.334309 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-62504017-8bc4-4541-99b7-a1ffb705e0f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c5c9bc2860a47568ab95d5b460137b6'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:22:57.334309 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-62504017-8bc4-4541-99b7-a1ffb705e0f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c5c9bc2860a47568ab95d5b460137b6'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:22:57.400261 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6c3d8f-8deb-48a1-8343-882e35bb9505 admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:22:57.401734 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-5e6c3d8f-8deb-48a1-8343-882e35bb9505 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:22:57.407734 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-62504017-8bc4-4541-99b7-a1ffb705e0f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c5c9bc2860a47568ab95d5b460137b6'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:22:57.415094 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 900/1783] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 01:22:56 2022] DELETE /identity/v3/users/1c5c9bc2860a47568ab95d5b460137b6 => generated 0 bytes in 486 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:22:57.421114 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6c3d8f-8deb-48a1-8343-882e35bb9505 admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:22:57.467764 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:22:57.483479 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198730ca-94fc-4759-81b3-ccb1efc7129e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:22:57.489949 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198730ca-94fc-4759-81b3-ccb1efc7129e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:57.489949 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198730ca-94fc-4759-81b3-ccb1efc7129e admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:57.489949 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198730ca-94fc-4759-81b3-ccb1efc7129e admin admin] PATH_INFO: `/v3/users/63cc85f4861948e79c90753f4ce2b4e9` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:57.489949 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198730ca-94fc-4759-81b3-ccb1efc7129e admin admin] RBAC: Authorizing `identity:delete_user(user_id=63cc85f4861948e79c90753f4ce2b4e9)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:22:57.490311 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198730ca-94fc-4759-81b3-ccb1efc7129e admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:22:57.726522 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-198730ca-94fc-4759-81b3-ccb1efc7129e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '63cc85f4861948e79c90753f4ce2b4e9'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:22:57.775291 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-198730ca-94fc-4759-81b3-ccb1efc7129e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '63cc85f4861948e79c90753f4ce2b4e9'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:22:57.775967 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-198730ca-94fc-4759-81b3-ccb1efc7129e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '63cc85f4861948e79c90753f4ce2b4e9'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:22:57.783946 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-198730ca-94fc-4759-81b3-ccb1efc7129e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '63cc85f4861948e79c90753f4ce2b4e9'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:22:57.784608 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-198730ca-94fc-4759-81b3-ccb1efc7129e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '63cc85f4861948e79c90753f4ce2b4e9'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:22:57.804710 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 884/1784] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 01:22:57 2022] DELETE /identity/v3/users/63cc85f4861948e79c90753f4ce2b4e9 => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:22:57.855433 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62504017-8bc4-4541-99b7-a1ffb705e0f0 admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:22:57.857045 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-62504017-8bc4-4541-99b7-a1ffb705e0f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:22:57.900859 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62504017-8bc4-4541-99b7-a1ffb705e0f0 admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:22:57.960552 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15c4d26-4b0c-4adb-be24-059af4d3d997 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:22:57.961735 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15c4d26-4b0c-4adb-be24-059af4d3d997 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:57.961735 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15c4d26-4b0c-4adb-be24-059af4d3d997 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:57.961955 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15c4d26-4b0c-4adb-be24-059af4d3d997 admin admin] PATH_INFO: `/v3/projects/c3696d906fb44760800af90cffe3abfa` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:57.963620 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15c4d26-4b0c-4adb-be24-059af4d3d997 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c3696d906fb44760800af90cffe3abfa)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:22:57.965382 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15c4d26-4b0c-4adb-be24-059af4d3d997 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:22:58.057162 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198730ca-94fc-4759-81b3-ccb1efc7129e admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:22:58.057162 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-198730ca-94fc-4759-81b3-ccb1efc7129e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:22:58.075837 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198730ca-94fc-4759-81b3-ccb1efc7129e admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:22:58.086444 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34e80b5-4784-4c75-bde1-e1802f77d8cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:22:58.087181 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34e80b5-4784-4c75-bde1-e1802f77d8cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:58.087606 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34e80b5-4784-4c75-bde1-e1802f77d8cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:58.088022 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34e80b5-4784-4c75-bde1-e1802f77d8cb admin admin] PATH_INFO: `/v3/projects/d4010fc9e21948adb150a09384956241` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:58.089275 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34e80b5-4784-4c75-bde1-e1802f77d8cb admin admin] RBAC: Authorizing `identity:delete_project(project_id=d4010fc9e21948adb150a09384956241)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:22:58.091281 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34e80b5-4784-4c75-bde1-e1802f77d8cb admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:22:58.114915 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-b15c4d26-4b0c-4adb-be24-059af4d3d997 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c3696d906fb44760800af90cffe3abfa', 'request_id': 'req-b15c4d26-4b0c-4adb-be24-059af4d3d997'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:22:58.153032 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-b15c4d26-4b0c-4adb-be24-059af4d3d997 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c3696d906fb44760800af90cffe3abfa', 'request_id': 'req-b15c4d26-4b0c-4adb-be24-059af4d3d997'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:22:58.257265 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-b15c4d26-4b0c-4adb-be24-059af4d3d997 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c3696d906fb44760800af90cffe3abfa', 'request_id': 'req-b15c4d26-4b0c-4adb-be24-059af4d3d997'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:22:58.266872 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-e34e80b5-4784-4c75-bde1-e1802f77d8cb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd4010fc9e21948adb150a09384956241', 'request_id': 'req-e34e80b5-4784-4c75-bde1-e1802f77d8cb'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:22:58.274398 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-e34e80b5-4784-4c75-bde1-e1802f77d8cb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd4010fc9e21948adb150a09384956241', 'request_id': 'req-e34e80b5-4784-4c75-bde1-e1802f77d8cb'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:22:58.276915 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-b15c4d26-4b0c-4adb-be24-059af4d3d997 admin admin] The token cache is being invalidate because project c3696d906fb44760800af90cffe3abfa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81750) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 01:22:58.277435 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-b15c4d26-4b0c-4adb-be24-059af4d3d997 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:22:58.281121 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-b15c4d26-4b0c-4adb-be24-059af4d3d997 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:22:58.281918 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-e34e80b5-4784-4c75-bde1-e1802f77d8cb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd4010fc9e21948adb150a09384956241', 'request_id': 'req-e34e80b5-4784-4c75-bde1-e1802f77d8cb'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:22:58.283516 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 901/1785] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 01:22:57 2022] DELETE /identity/v3/projects/c3696d906fb44760800af90cffe3abfa => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:22:58.288786 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641c909c-8707-4a21-bb42-87379f677edc None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:58.289565 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641c909c-8707-4a21-bb42-87379f677edc None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:58.289801 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641c909c-8707-4a21-bb42-87379f677edc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:58.333530 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-641c909c-8707-4a21-bb42-87379f677edc None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:22:58.335782 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-641c909c-8707-4a21-bb42-87379f677edc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:22:58.420588 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-e34e80b5-4784-4c75-bde1-e1802f77d8cb admin admin] The token cache is being invalidate because project d4010fc9e21948adb150a09384956241 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81749) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 01:22:58.428184 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 902/1786] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:22:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 01:22:58.433599 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-e34e80b5-4784-4c75-bde1-e1802f77d8cb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:22:58.433735 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36791102-2bf4-464b-903e-e79092115244 None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:58.433735 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36791102-2bf4-464b-903e-e79092115244 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:58.433735 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36791102-2bf4-464b-903e-e79092115244 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:58.433975 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-e34e80b5-4784-4c75-bde1-e1802f77d8cb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:22:58.434947 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 885/1787] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 01:22:58 2022] DELETE /identity/v3/projects/d4010fc9e21948adb150a09384956241 => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 01:22:58.441843 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34e80b5-4784-4c75-bde1-e1802f77d8cb admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:22:58.445628 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-e34e80b5-4784-4c75-bde1-e1802f77d8cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:22:58.477154 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34e80b5-4784-4c75-bde1-e1802f77d8cb admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:22:58.491516 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27830c1-02b4-471f-9501-3a04ce54ba1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:22:58.492385 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27830c1-02b4-471f-9501-3a04ce54ba1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:58.492799 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27830c1-02b4-471f-9501-3a04ce54ba1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:58.493263 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27830c1-02b4-471f-9501-3a04ce54ba1a admin admin] PATH_INFO: `/v3/users/13c39c33f32e44c9b3647028cf0b5b19` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:58.494709 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27830c1-02b4-471f-9501-3a04ce54ba1a admin admin] RBAC: Authorizing `identity:delete_user(user_id=13c39c33f32e44c9b3647028cf0b5b19)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:22:58.498291 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27830c1-02b4-471f-9501-3a04ce54ba1a admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:22:58.501462 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-36791102-2bf4-464b-903e-e79092115244 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:22:58.507261 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-36791102-2bf4-464b-903e-e79092115244 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:22:58.537603 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 903/1788] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:22:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:22:58.541932 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36791102-2bf4-464b-903e-e79092115244 None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:22:58.556955 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36791102-2bf4-464b-903e-e79092115244 None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:22:58.562225 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a1debe-85e4-4260-9a51-c6a9dfe82399 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:22:58.563954 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a1debe-85e4-4260-9a51-c6a9dfe82399 admin admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:58.564458 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a1debe-85e4-4260-9a51-c6a9dfe82399 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:58.565043 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a1debe-85e4-4260-9a51-c6a9dfe82399 admin admin] PATH_INFO: `/v3/domains` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:58.565927 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a1debe-85e4-4260-9a51-c6a9dfe82399 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:22:58.566446 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a1debe-85e4-4260-9a51-c6a9dfe82399 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:22:58.568366 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a1debe-85e4-4260-9a51-c6a9dfe82399 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:22:58.586013 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 904/1789] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Aug 31 01:22:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:22:58.590652 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a1debe-85e4-4260-9a51-c6a9dfe82399 admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:22:58.604894 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a1debe-85e4-4260-9a51-c6a9dfe82399 admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:22:58.615254 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a19547-740a-4a60-b5d4-bedcf48c72e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:22:58.616584 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a19547-740a-4a60-b5d4-bedcf48c72e7 admin admin] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:58.616969 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a19547-740a-4a60-b5d4-bedcf48c72e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:58.617252 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a19547-740a-4a60-b5d4-bedcf48c72e7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:58.618023 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a19547-740a-4a60-b5d4-bedcf48c72e7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:22:58.620364 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a19547-740a-4a60-b5d4-bedcf48c72e7 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:22:58.667091 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:22:58.739293 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 905/1790] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 01:22:58 2022] POST /identity/v3/projects => generated 391 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:22:58.743998 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a19547-740a-4a60-b5d4-bedcf48c72e7 admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:22:58.781605 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-e27830c1-02b4-471f-9501-3a04ce54ba1a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '13c39c33f32e44c9b3647028cf0b5b19'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:22:58.816649 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a19547-740a-4a60-b5d4-bedcf48c72e7 admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:22:58.826379 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e364d2-9992-4935-993a-1375a6850cce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:22:58.827023 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e364d2-9992-4935-993a-1375a6850cce admin admin] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:58.827306 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e364d2-9992-4935-993a-1375a6850cce admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:58.827650 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e364d2-9992-4935-993a-1375a6850cce admin admin] PATH_INFO: `/v3/users` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:58.828296 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e364d2-9992-4935-993a-1375a6850cce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:22:58.830852 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e364d2-9992-4935-993a-1375a6850cce admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:22:58.838110 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-e27830c1-02b4-471f-9501-3a04ce54ba1a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '13c39c33f32e44c9b3647028cf0b5b19'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:22:58.838692 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-e27830c1-02b4-471f-9501-3a04ce54ba1a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '13c39c33f32e44c9b3647028cf0b5b19'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:22:58.848096 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-e27830c1-02b4-471f-9501-3a04ce54ba1a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '13c39c33f32e44c9b3647028cf0b5b19'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:22:58.848240 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-e27830c1-02b4-471f-9501-3a04ce54ba1a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '13c39c33f32e44c9b3647028cf0b5b19'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:22:58.867519 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 886/1791] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 01:22:58 2022] DELETE /identity/v3/users/13c39c33f32e44c9b3647028cf0b5b19 => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:22:58.972850 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 906/1792] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 01:22:58 2022] POST /identity/v3/users => generated 296 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 01:22:58.974623 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27830c1-02b4-471f-9501-3a04ce54ba1a admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:22:58.976277 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-e27830c1-02b4-471f-9501-3a04ce54ba1a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:22:59.083097 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27830c1-02b4-471f-9501-3a04ce54ba1a admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:22:59.089495 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9adf789-4d52-470c-8328-795bfe0975c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:22:59.090318 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9adf789-4d52-470c-8328-795bfe0975c1 admin admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:59.090742 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9adf789-4d52-470c-8328-795bfe0975c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:59.091185 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9adf789-4d52-470c-8328-795bfe0975c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:59.091972 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9adf789-4d52-470c-8328-795bfe0975c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:22:59.093866 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9adf789-4d52-470c-8328-795bfe0975c1 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:22:59.122840 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e364d2-9992-4935-993a-1375a6850cce admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:22:59.132576 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 887/1793] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:22:58 2022] GET /identity/v3/roles => generated 1420 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:22:59.135047 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e364d2-9992-4935-993a-1375a6850cce admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:22:59.140223 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212eda2d-0c11-47fe-8442-e3bcb49b7d4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:22:59.140496 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212eda2d-0c11-47fe-8442-e3bcb49b7d4c admin admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:59.140872 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212eda2d-0c11-47fe-8442-e3bcb49b7d4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:59.141285 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212eda2d-0c11-47fe-8442-e3bcb49b7d4c admin admin] PATH_INFO: `/v3/roles` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:59.141946 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212eda2d-0c11-47fe-8442-e3bcb49b7d4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:22:59.143783 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212eda2d-0c11-47fe-8442-e3bcb49b7d4c admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:22:59.159419 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 907/1794] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:22:59 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:22:59.169117 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9adf789-4d52-470c-8328-795bfe0975c1 admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:22:59.176446 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9adf789-4d52-470c-8328-795bfe0975c1 admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:22:59.182126 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452210f4-c67f-4882-b4fc-cb188196fdc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:22:59.182934 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452210f4-c67f-4882-b4fc-cb188196fdc7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:59.183322 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452210f4-c67f-4882-b4fc-cb188196fdc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:59.183749 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452210f4-c67f-4882-b4fc-cb188196fdc7 admin admin] PATH_INFO: `/v3/projects/41c0ed31afdf4211beb6f2640905d764/users/68ad33b4f8054e5b9fc4b04c472d8fc6/roles/d0b9768f4d014766b9326387f2e11f34` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:59.196254 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212eda2d-0c11-47fe-8442-e3bcb49b7d4c admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:22:59.198716 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-212eda2d-0c11-47fe-8442-e3bcb49b7d4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:22:59.224817 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212eda2d-0c11-47fe-8442-e3bcb49b7d4c admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:22:59.232505 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452210f4-c67f-4882-b4fc-cb188196fdc7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=41c0ed31afdf4211beb6f2640905d764, user_id=68ad33b4f8054e5b9fc4b04c472d8fc6, role_id=d0b9768f4d014766b9326387f2e11f34)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:22:59.232677 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1333ccd-f7d8-40fc-96ac-5fef82683a1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:22:59.233467 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1333ccd-f7d8-40fc-96ac-5fef82683a1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:59.233785 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1333ccd-f7d8-40fc-96ac-5fef82683a1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:59.234043 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1333ccd-f7d8-40fc-96ac-5fef82683a1a admin admin] PATH_INFO: `/v3/projects/53be4396b4934ce9ac8a4e5cdd81333b` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:59.235704 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452210f4-c67f-4882-b4fc-cb188196fdc7 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:22:59.235965 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1333ccd-f7d8-40fc-96ac-5fef82683a1a admin admin] RBAC: Authorizing `identity:delete_project(project_id=53be4396b4934ce9ac8a4e5cdd81333b)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:22:59.239647 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1333ccd-f7d8-40fc-96ac-5fef82683a1a admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:22:59.292030 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 888/1795] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 01:22:59 2022] PUT /identity/v3/projects/41c0ed31afdf4211beb6f2640905d764/users/68ad33b4f8054e5b9fc4b04c472d8fc6/roles/d0b9768f4d014766b9326387f2e11f34 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:22:59.302186 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d9b075-ab64-4fb6-af8b-89c6a448aa08 None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:59.302479 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d9b075-ab64-4fb6-af8b-89c6a448aa08 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:59.302761 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d9b075-ab64-4fb6-af8b-89c6a448aa08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:59.335947 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-75d9b075-ab64-4fb6-af8b-89c6a448aa08 None None] MFA Rules not processed for user `68ad33b4f8054e5b9fc4b04c472d8fc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:22:59.341678 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-75d9b075-ab64-4fb6-af8b-89c6a448aa08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:22:59.369968 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-f1333ccd-f7d8-40fc-96ac-5fef82683a1a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '53be4396b4934ce9ac8a4e5cdd81333b', 'request_id': 'req-f1333ccd-f7d8-40fc-96ac-5fef82683a1a'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:22:59.376442 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-f1333ccd-f7d8-40fc-96ac-5fef82683a1a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '53be4396b4934ce9ac8a4e5cdd81333b', 'request_id': 'req-f1333ccd-f7d8-40fc-96ac-5fef82683a1a'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:22:59.410832 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 889/1796] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:22:59 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 01:22:59.415685 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d9b075-ab64-4fb6-af8b-89c6a448aa08 None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:22:59.498890 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-f1333ccd-f7d8-40fc-96ac-5fef82683a1a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '53be4396b4934ce9ac8a4e5cdd81333b', 'request_id': 'req-f1333ccd-f7d8-40fc-96ac-5fef82683a1a'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:22:59.527776 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-f1333ccd-f7d8-40fc-96ac-5fef82683a1a admin admin] The token cache is being invalidate because project 53be4396b4934ce9ac8a4e5cdd81333b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81750) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 01:22:59.528313 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-f1333ccd-f7d8-40fc-96ac-5fef82683a1a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:22:59.530004 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-f1333ccd-f7d8-40fc-96ac-5fef82683a1a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:22:59.532717 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 908/1797] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 01:22:59 2022] DELETE /identity/v3/projects/53be4396b4934ce9ac8a4e5cdd81333b => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:22:59.536804 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d9b075-ab64-4fb6-af8b-89c6a448aa08 None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:22:59.537977 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bf3044-99ea-4db3-9e87-c575b6fe0038 None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:59.538477 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bf3044-99ea-4db3-9e87-c575b6fe0038 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:59.538958 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bf3044-99ea-4db3-9e87-c575b6fe0038 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:59.570755 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-e9bf3044-99ea-4db3-9e87-c575b6fe0038 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:22:59.572327 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-e9bf3044-99ea-4db3-9e87-c575b6fe0038 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:22:59.668334 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 909/1798] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:22:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 01:22:59.676667 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a8f84e-3f32-405c-9a6b-943d489902f0 None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:59.677280 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a8f84e-3f32-405c-9a6b-943d489902f0 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:59.681093 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a8f84e-3f32-405c-9a6b-943d489902f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:59.683498 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c98fa3b-0206-4bd1-b224-febd225d52bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:22:59.684232 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c98fa3b-0206-4bd1-b224-febd225d52bf admin admin] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:59.684614 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c98fa3b-0206-4bd1-b224-febd225d52bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:59.684959 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c98fa3b-0206-4bd1-b224-febd225d52bf admin admin] PATH_INFO: `/v3/projects` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:59.685667 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c98fa3b-0206-4bd1-b224-febd225d52bf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:22:59.690292 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c98fa3b-0206-4bd1-b224-febd225d52bf admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:22:59.713667 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-18a8f84e-3f32-405c-9a6b-943d489902f0 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:22:59.714924 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-18a8f84e-3f32-405c-9a6b-943d489902f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:22:59.726527 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 910/1799] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:22:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:22:59.732773 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a8f84e-3f32-405c-9a6b-943d489902f0 None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:22:59.745268 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a8f84e-3f32-405c-9a6b-943d489902f0 None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:22:59.754687 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e74768-74d8-4eef-b915-08b1058c3e88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:22:59.754687 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e74768-74d8-4eef-b915-08b1058c3e88 admin admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:59.754687 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e74768-74d8-4eef-b915-08b1058c3e88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:59.754687 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e74768-74d8-4eef-b915-08b1058c3e88 admin admin] PATH_INFO: `/v3/domains` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:59.755101 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e74768-74d8-4eef-b915-08b1058c3e88 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:22:59.755101 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e74768-74d8-4eef-b915-08b1058c3e88 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:22:59.757072 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e74768-74d8-4eef-b915-08b1058c3e88 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:22:59.781723 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 911/1800] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Aug 31 01:22:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 01:22:59.790449 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e74768-74d8-4eef-b915-08b1058c3e88 admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:22:59.801051 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e74768-74d8-4eef-b915-08b1058c3e88 admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:22:59.807139 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20608dbe-65a9-422e-aa25-bceb9cfa19f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:22:59.808745 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20608dbe-65a9-422e-aa25-bceb9cfa19f5 admin admin] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:22:59.808745 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20608dbe-65a9-422e-aa25-bceb9cfa19f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:22:59.808745 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20608dbe-65a9-422e-aa25-bceb9cfa19f5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:22:59.809501 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20608dbe-65a9-422e-aa25-bceb9cfa19f5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:22:59.811423 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20608dbe-65a9-422e-aa25-bceb9cfa19f5 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:22:59.837347 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 890/1801] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 01:22:59 2022] POST /identity/v3/projects => generated 389 bytes in 423 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:22:59.841743 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c98fa3b-0206-4bd1-b224-febd225d52bf admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:22:59.847767 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-5c98fa3b-0206-4bd1-b224-febd225d52bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:22:59.907433 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=74,keystone:DELETE=24,keystone:INSERT=2 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:22:59.945172 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 912/1802] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 01:22:59 2022] POST /identity/v3/projects => generated 365 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 01:22:59.946187 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20608dbe-65a9-422e-aa25-bceb9cfa19f5 admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:22:59.947884 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-20608dbe-65a9-422e-aa25-bceb9cfa19f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:22:59.987437 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c98fa3b-0206-4bd1-b224-febd225d52bf admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:00.030266 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20608dbe-65a9-422e-aa25-bceb9cfa19f5 admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:00.036865 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ade716-5916-455a-975d-8b387605cc42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:00.038320 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ade716-5916-455a-975d-8b387605cc42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:00.038686 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ade716-5916-455a-975d-8b387605cc42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:00.039008 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ade716-5916-455a-975d-8b387605cc42 admin admin] PATH_INFO: `/v3/users/04a03327e5f342dc9dbf7a5d638a61c3` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:00.040224 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ade716-5916-455a-975d-8b387605cc42 admin admin] RBAC: Authorizing `identity:delete_user(user_id=04a03327e5f342dc9dbf7a5d638a61c3)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:00.042210 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ade716-5916-455a-975d-8b387605cc42 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:00.082851 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84bf629-dfd7-4b0f-9e05-76c570462b97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:00.083520 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84bf629-dfd7-4b0f-9e05-76c570462b97 admin admin] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:00.084053 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84bf629-dfd7-4b0f-9e05-76c570462b97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:00.086600 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84bf629-dfd7-4b0f-9e05-76c570462b97 admin admin] PATH_INFO: `/v3/users` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:00.086600 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84bf629-dfd7-4b0f-9e05-76c570462b97 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:00.087195 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84bf629-dfd7-4b0f-9e05-76c570462b97 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:00.167491 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 891/1803] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 01:22:59 2022] POST /identity/v3/users => generated 301 bytes in 326 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:23:00.168366 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84bf629-dfd7-4b0f-9e05-76c570462b97 admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:00.179148 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84bf629-dfd7-4b0f-9e05-76c570462b97 admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:00.183708 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2704f9a3-22f4-4204-9a51-551c14423e6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:00.184307 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2704f9a3-22f4-4204-9a51-551c14423e6f admin admin] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:00.184590 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2704f9a3-22f4-4204-9a51-551c14423e6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:00.184902 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2704f9a3-22f4-4204-9a51-551c14423e6f admin admin] PATH_INFO: `/v3/users` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:00.185532 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2704f9a3-22f4-4204-9a51-551c14423e6f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:00.187435 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2704f9a3-22f4-4204-9a51-551c14423e6f admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:00.266208 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-e5ade716-5916-455a-975d-8b387605cc42 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '04a03327e5f342dc9dbf7a5d638a61c3'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:00.279984 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-e5ade716-5916-455a-975d-8b387605cc42 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '04a03327e5f342dc9dbf7a5d638a61c3'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:00.280769 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-e5ade716-5916-455a-975d-8b387605cc42 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '04a03327e5f342dc9dbf7a5d638a61c3'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:00.288286 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-e5ade716-5916-455a-975d-8b387605cc42 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '04a03327e5f342dc9dbf7a5d638a61c3'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:00.315995 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 892/1804] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 01:23:00 2022] POST /identity/v3/users => generated 283 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:23:00.315995 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2704f9a3-22f4-4204-9a51-551c14423e6f admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:00.315995 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-2704f9a3-22f4-4204-9a51-551c14423e6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:23:00.326521 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2704f9a3-22f4-4204-9a51-551c14423e6f admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:00.331331 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a3bfaa-cf73-43a6-84bd-6c718b9e405e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:00.331904 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a3bfaa-cf73-43a6-84bd-6c718b9e405e admin admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:00.332183 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a3bfaa-cf73-43a6-84bd-6c718b9e405e admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:00.332454 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a3bfaa-cf73-43a6-84bd-6c718b9e405e admin admin] PATH_INFO: `/v3/roles` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:00.333141 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a3bfaa-cf73-43a6-84bd-6c718b9e405e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:00.335256 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a3bfaa-cf73-43a6-84bd-6c718b9e405e admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:00.347771 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 893/1805] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:23:00 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:23:00.348624 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a3bfaa-cf73-43a6-84bd-6c718b9e405e admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:00.355875 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-12a3bfaa-cf73-43a6-84bd-6c718b9e405e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:23:00.364162 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a3bfaa-cf73-43a6-84bd-6c718b9e405e admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:00.369140 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bfd847-9051-433d-be6c-5fd4649b4235 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:00.369688 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bfd847-9051-433d-be6c-5fd4649b4235 admin admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:00.370065 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bfd847-9051-433d-be6c-5fd4649b4235 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:00.370349 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bfd847-9051-433d-be6c-5fd4649b4235 admin admin] PATH_INFO: `/v3/roles` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:00.371052 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bfd847-9051-433d-be6c-5fd4649b4235 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:00.372641 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bfd847-9051-433d-be6c-5fd4649b4235 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:00.376663 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-e5ade716-5916-455a-975d-8b387605cc42 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '04a03327e5f342dc9dbf7a5d638a61c3'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:00.383657 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 913/1806] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 01:22:59 2022] DELETE /identity/v3/users/04a03327e5f342dc9dbf7a5d638a61c3 => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:23:00.387660 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 894/1807] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:23:00 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:23:00.387818 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ade716-5916-455a-975d-8b387605cc42 admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:00.388312 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6867b9-1fa7-4938-aa23-c4aa8cfed0af None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:00.388849 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6867b9-1fa7-4938-aa23-c4aa8cfed0af None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:00.389020 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6867b9-1fa7-4938-aa23-c4aa8cfed0af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:00.418419 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-1f6867b9-1fa7-4938-aa23-c4aa8cfed0af None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:23:00.420079 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-1f6867b9-1fa7-4938-aa23-c4aa8cfed0af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:23:00.529691 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 895/1808] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:23:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 01:23:00.530458 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6867b9-1fa7-4938-aa23-c4aa8cfed0af None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:00.544715 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6867b9-1fa7-4938-aa23-c4aa8cfed0af None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:00.546051 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ade716-5916-455a-975d-8b387605cc42 admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:00.551295 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e2f49b-8800-4537-bd12-ffff2bfaf8e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:00.552029 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc9d497-c56b-4a69-b5fc-2784b53d3557 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:00.552155 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e2f49b-8800-4537-bd12-ffff2bfaf8e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:00.552474 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e2f49b-8800-4537-bd12-ffff2bfaf8e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:00.552665 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc9d497-c56b-4a69-b5fc-2784b53d3557 admin admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:00.552835 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e2f49b-8800-4537-bd12-ffff2bfaf8e2 admin admin] PATH_INFO: `/v3/projects/b46c20215e15449daeea6291edeceb59/users/b131c52388134817a66860e60bc363a7/roles/d009411496b8452b88e77ffb8db015a7` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:00.553148 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc9d497-c56b-4a69-b5fc-2784b53d3557 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:00.553418 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc9d497-c56b-4a69-b5fc-2784b53d3557 admin admin] PATH_INFO: `/v3/roles` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:00.554090 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc9d497-c56b-4a69-b5fc-2784b53d3557 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:00.555855 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc9d497-c56b-4a69-b5fc-2784b53d3557 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:00.567103 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 896/1809] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:23:00 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:23:00.568005 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc9d497-c56b-4a69-b5fc-2784b53d3557 admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:00.570203 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-8cc9d497-c56b-4a69-b5fc-2784b53d3557 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:23:00.576671 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e2f49b-8800-4537-bd12-ffff2bfaf8e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b46c20215e15449daeea6291edeceb59, user_id=b131c52388134817a66860e60bc363a7, role_id=d009411496b8452b88e77ffb8db015a7)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:00.581436 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e2f49b-8800-4537-bd12-ffff2bfaf8e2 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:00.637527 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc9d497-c56b-4a69-b5fc-2784b53d3557 admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:00.641276 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa86e74-c0e3-4e84-b46b-5b66b1717ace service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0817c3320d5d41d8bba0550c8ae8dd60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:00.641898 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa86e74-c0e3-4e84-b46b-5b66b1717ace service neutron] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:00.642178 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa86e74-c0e3-4e84-b46b-5b66b1717ace service neutron] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:00.642449 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa86e74-c0e3-4e84-b46b-5b66b1717ace service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:00.658008 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa86e74-c0e3-4e84-b46b-5b66b1717ace service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:00.659523 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa86e74-c0e3-4e84-b46b-5b66b1717ace service neutron] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:00.676961 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 897/1810] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:23:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:23:00.678763 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa86e74-c0e3-4e84-b46b-5b66b1717ace service neutron] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:00.695657 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa86e74-c0e3-4e84-b46b-5b66b1717ace service neutron] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:00.702711 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d509558-904f-4a8f-8070-9b448f3db8a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:00.703658 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d509558-904f-4a8f-8070-9b448f3db8a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:00.704319 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d509558-904f-4a8f-8070-9b448f3db8a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:00.704847 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d509558-904f-4a8f-8070-9b448f3db8a3 admin admin] PATH_INFO: `/v3/projects/52b8c1fb7247417b9440c0a4738a5a5b/users/af0c6ebaa47447709fd94b142923b1ee/roles/d0b9768f4d014766b9326387f2e11f34` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:00.715205 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 914/1811] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 01:23:00 2022] PUT /identity/v3/projects/b46c20215e15449daeea6291edeceb59/users/b131c52388134817a66860e60bc363a7/roles/d009411496b8452b88e77ffb8db015a7 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:23:00.717621 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e2f49b-8800-4537-bd12-ffff2bfaf8e2 admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:00.727217 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d509558-904f-4a8f-8070-9b448f3db8a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=52b8c1fb7247417b9440c0a4738a5a5b, user_id=af0c6ebaa47447709fd94b142923b1ee, role_id=d0b9768f4d014766b9326387f2e11f34)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:00.730680 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d509558-904f-4a8f-8070-9b448f3db8a3 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:00.841172 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 898/1812] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 01:23:00 2022] PUT /identity/v3/projects/52b8c1fb7247417b9440c0a4738a5a5b/users/af0c6ebaa47447709fd94b142923b1ee/roles/d0b9768f4d014766b9326387f2e11f34 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:23:00.843255 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb0ddb9-261d-4e05-b528-90f7a3cb0269 None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:00.843852 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb0ddb9-261d-4e05-b528-90f7a3cb0269 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:00.844240 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb0ddb9-261d-4e05-b528-90f7a3cb0269 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:00.878616 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e2f49b-8800-4537-bd12-ffff2bfaf8e2 admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:00.900959 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-3cb0ddb9-261d-4e05-b528-90f7a3cb0269 None None] MFA Rules not processed for user `af0c6ebaa47447709fd94b142923b1ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:23:00.902255 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-3cb0ddb9-261d-4e05-b528-90f7a3cb0269 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:23:01.029351 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5117d7-0f42-49b1-9f53-59890ec9e9e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:01.030147 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5117d7-0f42-49b1-9f53-59890ec9e9e8 admin admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:01.030432 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5117d7-0f42-49b1-9f53-59890ec9e9e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:01.031030 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5117d7-0f42-49b1-9f53-59890ec9e9e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:01.031786 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5117d7-0f42-49b1-9f53-59890ec9e9e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:01.033499 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5117d7-0f42-49b1-9f53-59890ec9e9e8 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:01.046920 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 915/1813] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:23:00 2022] GET /identity/v3/roles => generated 1420 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:23:01.053162 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5117d7-0f42-49b1-9f53-59890ec9e9e8 admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:01.069444 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 899/1814] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:23:00 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 01:23:01.072248 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5117d7-0f42-49b1-9f53-59890ec9e9e8 admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:01.073863 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb0ddb9-261d-4e05-b528-90f7a3cb0269 None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:01.087080 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbff5f1-cb86-4eaf-82f8-bd62230bb0c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:01.087902 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbff5f1-cb86-4eaf-82f8-bd62230bb0c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:01.088319 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbff5f1-cb86-4eaf-82f8-bd62230bb0c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:01.088838 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbff5f1-cb86-4eaf-82f8-bd62230bb0c1 admin admin] PATH_INFO: `/v3/projects/b46c20215e15449daeea6291edeceb59/users/b131c52388134817a66860e60bc363a7/roles/d009411496b8452b88e77ffb8db015a7` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:01.092980 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbff5f1-cb86-4eaf-82f8-bd62230bb0c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b46c20215e15449daeea6291edeceb59, user_id=b131c52388134817a66860e60bc363a7, role_id=d009411496b8452b88e77ffb8db015a7)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:01.095774 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbff5f1-cb86-4eaf-82f8-bd62230bb0c1 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:01.100711 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb0ddb9-261d-4e05-b528-90f7a3cb0269 None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:01.113158 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316bf896-415a-42df-97a5-08af5fb5a5ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:01.113895 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316bf896-415a-42df-97a5-08af5fb5a5ed admin admin] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:01.114753 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316bf896-415a-42df-97a5-08af5fb5a5ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:01.115173 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316bf896-415a-42df-97a5-08af5fb5a5ed admin admin] PATH_INFO: `/v3/projects` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:01.116884 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316bf896-415a-42df-97a5-08af5fb5a5ed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:01.123361 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316bf896-415a-42df-97a5-08af5fb5a5ed admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:01.138316 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 916/1815] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 01:23:01 2022] PUT /identity/v3/projects/b46c20215e15449daeea6291edeceb59/users/b131c52388134817a66860e60bc363a7/roles/d009411496b8452b88e77ffb8db015a7 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:23:01.150609 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42c34c2-5545-4de5-8274-82ac317b45f6 None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:01.151080 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42c34c2-5545-4de5-8274-82ac317b45f6 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:01.151836 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42c34c2-5545-4de5-8274-82ac317b45f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:01.231247 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-d42c34c2-5545-4de5-8274-82ac317b45f6 None None] MFA Rules not processed for user `b131c52388134817a66860e60bc363a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:23:01.234472 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-d42c34c2-5545-4de5-8274-82ac317b45f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:23:01.334753 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 917/1816] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:23:01 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 01:23:01.346466 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42c34c2-5545-4de5-8274-82ac317b45f6 None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:01.348021 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-d42c34c2-5545-4de5-8274-82ac317b45f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:23:01.423682 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42c34c2-5545-4de5-8274-82ac317b45f6 None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:01.428895 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a19693-5d52-4ee4-9fd9-f6f4557fe846 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5c8fc69c518412ba1aad61a9f4f558c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:01.429552 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a19693-5d52-4ee4-9fd9-f6f4557fe846 service cinder] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:01.429840 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a19693-5d52-4ee4-9fd9-f6f4557fe846 service cinder] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:01.430118 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a19693-5d52-4ee4-9fd9-f6f4557fe846 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:01.440140 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a19693-5d52-4ee4-9fd9-f6f4557fe846 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:01.441617 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a19693-5d52-4ee4-9fd9-f6f4557fe846 service cinder] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:01.452185 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 918/1817] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:23:01 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 01:23:01.512566 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a19693-5d52-4ee4-9fd9-f6f4557fe846 service cinder] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:01.514157 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-29a19693-5d52-4ee4-9fd9-f6f4557fe846 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:23:01.533479 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a19693-5d52-4ee4-9fd9-f6f4557fe846 service cinder] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:01.545722 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdd65f6-c0ea-476e-9212-6cf9764730a3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5c8fc69c518412ba1aad61a9f4f558c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:01.546372 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdd65f6-c0ea-476e-9212-6cf9764730a3 service cinder] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:01.546654 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdd65f6-c0ea-476e-9212-6cf9764730a3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:01.547110 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdd65f6-c0ea-476e-9212-6cf9764730a3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:01.549176 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-fcdd65f6-c0ea-476e-9212-6cf9764730a3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:23:01.635981 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdd65f6-c0ea-476e-9212-6cf9764730a3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:01.640454 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdd65f6-c0ea-476e-9212-6cf9764730a3 service cinder] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:01.654296 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 919/1818] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:23:01 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 01:23:02.410446 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 900/1819] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 01:23:01 2022] POST /identity/v3/projects => generated 365 bytes in 1337 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:23:02.414814 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdd65f6-c0ea-476e-9212-6cf9764730a3 service cinder] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:02.570888 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdd65f6-c0ea-476e-9212-6cf9764730a3 service cinder] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:02.588733 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=90,keystone:INSERT=4,keystone:DELETE=6 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:23:02.672876 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5220e64c-618c-4061-8af1-50c614003b88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:02.678512 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4e88c4-1a5b-4a52-b23f-230f807539a3 None None] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:02.678512 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4e88c4-1a5b-4a52-b23f-230f807539a3 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:02.679204 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5220e64c-618c-4061-8af1-50c614003b88 admin admin] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:02.679204 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5220e64c-618c-4061-8af1-50c614003b88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:02.679369 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4e88c4-1a5b-4a52-b23f-230f807539a3 None None] PATH_INFO: `/` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:02.679453 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5220e64c-618c-4061-8af1-50c614003b88 admin admin] PATH_INFO: `/v3/users` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:02.679453 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5220e64c-618c-4061-8af1-50c614003b88 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:02.679627 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 901/1820] 10.222.0.20 () {54 vars in 921 bytes} [Wed Aug 31 01:23:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:23:02.679715 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5220e64c-618c-4061-8af1-50c614003b88 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:02.783798 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b817065f-938a-4427-a86c-3b4bcba126af None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:02.784924 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b817065f-938a-4427-a86c-3b4bcba126af None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:02.785048 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b817065f-938a-4427-a86c-3b4bcba126af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:02.791983 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 920/1821] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 01:23:02 2022] POST /identity/v3/users => generated 289 bytes in 377 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:23:02.799276 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5220e64c-618c-4061-8af1-50c614003b88 admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:02.816528 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5220e64c-618c-4061-8af1-50c614003b88 admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:02.829353 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de2680f-ef65-45e9-a089-7322b6a4139b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:02.830611 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de2680f-ef65-45e9-a089-7322b6a4139b admin admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:02.830896 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de2680f-ef65-45e9-a089-7322b6a4139b admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:02.831176 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de2680f-ef65-45e9-a089-7322b6a4139b admin admin] PATH_INFO: `/v3/roles` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:02.833418 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de2680f-ef65-45e9-a089-7322b6a4139b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:02.836337 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de2680f-ef65-45e9-a089-7322b6a4139b admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:02.837751 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-b817065f-938a-4427-a86c-3b4bcba126af None None] MFA Rules not processed for user `8744c9de803048f583d0317d01f6a540`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:23:02.840889 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-b817065f-938a-4427-a86c-3b4bcba126af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:23:02.856996 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 921/1822] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:23:02 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:23:02.864180 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de2680f-ef65-45e9-a089-7322b6a4139b admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:02.881773 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de2680f-ef65-45e9-a089-7322b6a4139b admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:02.893659 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f802f131-e08a-4c48-9a8c-035f6a84d5fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:02.894587 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f802f131-e08a-4c48-9a8c-035f6a84d5fa admin admin] REQUEST_METHOD: `PUT` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:02.897354 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f802f131-e08a-4c48-9a8c-035f6a84d5fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:02.897354 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f802f131-e08a-4c48-9a8c-035f6a84d5fa admin admin] PATH_INFO: `/v3/projects/523e95c967ac42809cb1b3799a238cb8/users/bbeedb790e734326a85a77f5a1b237b2/roles/d009411496b8452b88e77ffb8db015a7` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:02.921308 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f802f131-e08a-4c48-9a8c-035f6a84d5fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=523e95c967ac42809cb1b3799a238cb8, user_id=bbeedb790e734326a85a77f5a1b237b2, role_id=d009411496b8452b88e77ffb8db015a7)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:02.926084 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f802f131-e08a-4c48-9a8c-035f6a84d5fa admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:02.935318 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 902/1823] 10.222.0.20 () {58 vars in 1018 bytes} [Wed Aug 31 01:23:02 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 01:23:03.004995 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 922/1824] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 01:23:02 2022] PUT /identity/v3/projects/523e95c967ac42809cb1b3799a238cb8/users/bbeedb790e734326a85a77f5a1b237b2/roles/d009411496b8452b88e77ffb8db015a7 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 01:23:03.009565 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b817065f-938a-4427-a86c-3b4bcba126af None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:03.025991 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b817065f-938a-4427-a86c-3b4bcba126af None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:03.043299 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f802f131-e08a-4c48-9a8c-035f6a84d5fa admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:03.048227 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-f802f131-e08a-4c48-9a8c-035f6a84d5fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:23:03.249305 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45efe02-6e3b-4106-a907-dd854be66fce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:03.250795 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45efe02-6e3b-4106-a907-dd854be66fce admin admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:03.250795 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45efe02-6e3b-4106-a907-dd854be66fce admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:03.251042 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45efe02-6e3b-4106-a907-dd854be66fce admin admin] PATH_INFO: `/v3/roles` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:03.251495 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45efe02-6e3b-4106-a907-dd854be66fce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:03.254488 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45efe02-6e3b-4106-a907-dd854be66fce admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:03.255492 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f802f131-e08a-4c48-9a8c-035f6a84d5fa admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:03.278487 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 903/1825] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:23:03 2022] GET /identity/v3/roles => generated 1420 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:23:03.282612 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0eeef3-4a39-4e7d-8d72-d26eaabfcb46 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8744c9de803048f583d0317d01f6a540', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:03.283268 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0eeef3-4a39-4e7d-8d72-d26eaabfcb46 service nova] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:03.283887 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0eeef3-4a39-4e7d-8d72-d26eaabfcb46 service nova] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:03.283887 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0eeef3-4a39-4e7d-8d72-d26eaabfcb46 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:03.285265 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45efe02-6e3b-4106-a907-dd854be66fce admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:03.305914 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45efe02-6e3b-4106-a907-dd854be66fce admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:03.313458 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0eeef3-4a39-4e7d-8d72-d26eaabfcb46 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:03.315285 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0eeef3-4a39-4e7d-8d72-d26eaabfcb46 service nova] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:03.323273 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7106d0-1113-498e-9e99-8d96b62623b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:03.324285 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7106d0-1113-498e-9e99-8d96b62623b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:03.324643 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7106d0-1113-498e-9e99-8d96b62623b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:03.325018 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7106d0-1113-498e-9e99-8d96b62623b9 admin admin] PATH_INFO: `/v3/projects/523e95c967ac42809cb1b3799a238cb8/users/bbeedb790e734326a85a77f5a1b237b2/roles/d009411496b8452b88e77ffb8db015a7` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:03.327332 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7106d0-1113-498e-9e99-8d96b62623b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=523e95c967ac42809cb1b3799a238cb8, user_id=bbeedb790e734326a85a77f5a1b237b2, role_id=d009411496b8452b88e77ffb8db015a7)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:03.330620 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7106d0-1113-498e-9e99-8d96b62623b9 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:03.338651 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 923/1826] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:23:03 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:23:03.340550 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0eeef3-4a39-4e7d-8d72-d26eaabfcb46 service nova] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:03.343803 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-8c0eeef3-4a39-4e7d-8d72-d26eaabfcb46 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:23:03.364818 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 904/1827] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 01:23:03 2022] PUT /identity/v3/projects/523e95c967ac42809cb1b3799a238cb8/users/bbeedb790e734326a85a77f5a1b237b2/roles/d009411496b8452b88e77ffb8db015a7 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:23:03.364818 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17aa734e-fa23-49e0-9690-50ba2ef59622 None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:03.364818 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17aa734e-fa23-49e0-9690-50ba2ef59622 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:03.364818 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17aa734e-fa23-49e0-9690-50ba2ef59622 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:03.394070 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0eeef3-4a39-4e7d-8d72-d26eaabfcb46 service nova] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:03.425059 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-17aa734e-fa23-49e0-9690-50ba2ef59622 None None] MFA Rules not processed for user `bbeedb790e734326a85a77f5a1b237b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:23:03.426937 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-17aa734e-fa23-49e0-9690-50ba2ef59622 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:23:03.560762 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10733a1b-9a7f-4a39-b949-3544b752cbae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:03.563118 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10733a1b-9a7f-4a39-b949-3544b752cbae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:03.564067 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10733a1b-9a7f-4a39-b949-3544b752cbae admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:03.565768 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10733a1b-9a7f-4a39-b949-3544b752cbae admin admin] PATH_INFO: `/v3/projects/171940f7cda6497e93297c29bb1baae1` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:03.569459 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10733a1b-9a7f-4a39-b949-3544b752cbae admin admin] RBAC: Authorizing `identity:delete_project(project_id=171940f7cda6497e93297c29bb1baae1)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:03.576810 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10733a1b-9a7f-4a39-b949-3544b752cbae admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:03.616810 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:23:03.645874 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 905/1828] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:23:03 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 01:23:03.657435 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17aa734e-fa23-49e0-9690-50ba2ef59622 None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:03.756071 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17aa734e-fa23-49e0-9690-50ba2ef59622 None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:03.799142 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-10733a1b-9a7f-4a39-b949-3544b752cbae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '171940f7cda6497e93297c29bb1baae1', 'request_id': 'req-10733a1b-9a7f-4a39-b949-3544b752cbae'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:03.811036 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-10733a1b-9a7f-4a39-b949-3544b752cbae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '171940f7cda6497e93297c29bb1baae1', 'request_id': 'req-10733a1b-9a7f-4a39-b949-3544b752cbae'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:03.862209 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b78c83-8a7b-4ef8-83b6-696a1557e3d9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5c8fc69c518412ba1aad61a9f4f558c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:03.863492 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b78c83-8a7b-4ef8-83b6-696a1557e3d9 service cinder] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:03.863834 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b78c83-8a7b-4ef8-83b6-696a1557e3d9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:03.864322 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b78c83-8a7b-4ef8-83b6-696a1557e3d9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:03.918691 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-10733a1b-9a7f-4a39-b949-3544b752cbae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '171940f7cda6497e93297c29bb1baae1', 'request_id': 'req-10733a1b-9a7f-4a39-b949-3544b752cbae'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:03.939014 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-10733a1b-9a7f-4a39-b949-3544b752cbae admin admin] The token cache is being invalidate because project 171940f7cda6497e93297c29bb1baae1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81750) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 01:23:03.939465 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-10733a1b-9a7f-4a39-b949-3544b752cbae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:03.941285 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-10733a1b-9a7f-4a39-b949-3544b752cbae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:03.944861 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 924/1829] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 01:23:03 2022] DELETE /identity/v3/projects/171940f7cda6497e93297c29bb1baae1 => generated 0 bytes in 606 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:23:03.949831 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10733a1b-9a7f-4a39-b949-3544b752cbae admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:03.953823 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-10733a1b-9a7f-4a39-b949-3544b752cbae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:23:03.957902 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b78c83-8a7b-4ef8-83b6-696a1557e3d9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:03.959497 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b78c83-8a7b-4ef8-83b6-696a1557e3d9 service cinder] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:03.972127 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 906/1830] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:23:03 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:23:04.083684 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10733a1b-9a7f-4a39-b949-3544b752cbae admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:04.096197 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d9ce70-86c9-4afe-9472-7c3bba2c8c1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:04.097273 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d9ce70-86c9-4afe-9472-7c3bba2c8c1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:04.098290 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d9ce70-86c9-4afe-9472-7c3bba2c8c1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:04.098701 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d9ce70-86c9-4afe-9472-7c3bba2c8c1d admin admin] PATH_INFO: `/v3/users/d9d2efb56e2b48a68317275a0a8c115e` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:04.101254 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d9ce70-86c9-4afe-9472-7c3bba2c8c1d admin admin] RBAC: Authorizing `identity:delete_user(user_id=d9d2efb56e2b48a68317275a0a8c115e)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:04.105568 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d9ce70-86c9-4afe-9472-7c3bba2c8c1d admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:04.284431 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b78c83-8a7b-4ef8-83b6-696a1557e3d9 service cinder] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:04.294411 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b78c83-8a7b-4ef8-83b6-696a1557e3d9 service cinder] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:04.316754 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f839a7-fbdb-45d5-aba6-7d3e64162ef7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5c8fc69c518412ba1aad61a9f4f558c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:04.317445 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f839a7-fbdb-45d5-aba6-7d3e64162ef7 service cinder] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:04.317735 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f839a7-fbdb-45d5-aba6-7d3e64162ef7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:04.318013 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f839a7-fbdb-45d5-aba6-7d3e64162ef7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:04.321868 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-a7f839a7-fbdb-45d5-aba6-7d3e64162ef7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:23:04.371693 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-66d9ce70-86c9-4afe-9472-7c3bba2c8c1d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd9d2efb56e2b48a68317275a0a8c115e'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:04.399793 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-66d9ce70-86c9-4afe-9472-7c3bba2c8c1d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd9d2efb56e2b48a68317275a0a8c115e'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:04.403637 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-66d9ce70-86c9-4afe-9472-7c3bba2c8c1d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd9d2efb56e2b48a68317275a0a8c115e'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:04.415731 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-66d9ce70-86c9-4afe-9472-7c3bba2c8c1d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd9d2efb56e2b48a68317275a0a8c115e'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:04.416933 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f839a7-fbdb-45d5-aba6-7d3e64162ef7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:04.419725 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f839a7-fbdb-45d5-aba6-7d3e64162ef7 service cinder] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:04.437039 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 907/1831] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:23:04 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 01:23:04.498716 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-66d9ce70-86c9-4afe-9472-7c3bba2c8c1d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd9d2efb56e2b48a68317275a0a8c115e'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:04.505482 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 925/1832] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 01:23:03 2022] DELETE /identity/v3/users/d9d2efb56e2b48a68317275a0a8c115e => generated 0 bytes in 556 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:23:05.095554 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:INSERT=2,keystone:DELETE=8,keystone:SELECT=90 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:23:05.307009 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f839a7-fbdb-45d5-aba6-7d3e64162ef7 service cinder] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:05.342290 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f839a7-fbdb-45d5-aba6-7d3e64162ef7 service cinder] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:05.447116 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb94a16-ad33-4025-9d76-48b55ff7532f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:05.447761 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb94a16-ad33-4025-9d76-48b55ff7532f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:05.448039 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb94a16-ad33-4025-9d76-48b55ff7532f admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:05.448336 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb94a16-ad33-4025-9d76-48b55ff7532f admin admin] PATH_INFO: `/v3/projects/9a29535d173c49c398e2bc713df03755` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:05.449525 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb94a16-ad33-4025-9d76-48b55ff7532f admin admin] RBAC: Authorizing `identity:delete_project(project_id=9a29535d173c49c398e2bc713df03755)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:05.451519 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb94a16-ad33-4025-9d76-48b55ff7532f admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:05.611212 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-6cb94a16-ad33-4025-9d76-48b55ff7532f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9a29535d173c49c398e2bc713df03755', 'request_id': 'req-6cb94a16-ad33-4025-9d76-48b55ff7532f'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:05.631568 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-6cb94a16-ad33-4025-9d76-48b55ff7532f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9a29535d173c49c398e2bc713df03755', 'request_id': 'req-6cb94a16-ad33-4025-9d76-48b55ff7532f'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:05.639193 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-6cb94a16-ad33-4025-9d76-48b55ff7532f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9a29535d173c49c398e2bc713df03755', 'request_id': 'req-6cb94a16-ad33-4025-9d76-48b55ff7532f'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:05.821266 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-6cb94a16-ad33-4025-9d76-48b55ff7532f admin admin] The token cache is being invalidate because project 9a29535d173c49c398e2bc713df03755 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81749) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 01:23:05.821266 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-6cb94a16-ad33-4025-9d76-48b55ff7532f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:05.821266 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-6cb94a16-ad33-4025-9d76-48b55ff7532f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:05.821266 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 908/1833] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 01:23:05 2022] DELETE /identity/v3/projects/9a29535d173c49c398e2bc713df03755 => generated 0 bytes in 510 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:23:05.822118 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d9ce70-86c9-4afe-9472-7c3bba2c8c1d admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:05.822118 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-66d9ce70-86c9-4afe-9472-7c3bba2c8c1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:23:05.856889 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d9ce70-86c9-4afe-9472-7c3bba2c8c1d admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:05.953099 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b5fc03-c00c-456d-ab53-5ff3b954bf78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:05.953960 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b5fc03-c00c-456d-ab53-5ff3b954bf78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:05.957485 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b5fc03-c00c-456d-ab53-5ff3b954bf78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:05.957899 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b5fc03-c00c-456d-ab53-5ff3b954bf78 admin admin] PATH_INFO: `/v3/users/07e6e939c82c4864806c0d4df4e93a86` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:05.960400 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b5fc03-c00c-456d-ab53-5ff3b954bf78 admin admin] RBAC: Authorizing `identity:delete_user(user_id=07e6e939c82c4864806c0d4df4e93a86)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:05.965140 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b5fc03-c00c-456d-ab53-5ff3b954bf78 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:06.217680 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-a6b5fc03-c00c-456d-ab53-5ff3b954bf78 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '07e6e939c82c4864806c0d4df4e93a86'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:06.227238 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:23:06.230883 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-a6b5fc03-c00c-456d-ab53-5ff3b954bf78 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '07e6e939c82c4864806c0d4df4e93a86'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:06.231589 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-a6b5fc03-c00c-456d-ab53-5ff3b954bf78 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '07e6e939c82c4864806c0d4df4e93a86'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:06.238775 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-a6b5fc03-c00c-456d-ab53-5ff3b954bf78 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '07e6e939c82c4864806c0d4df4e93a86'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:06.310600 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-a6b5fc03-c00c-456d-ab53-5ff3b954bf78 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '07e6e939c82c4864806c0d4df4e93a86'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:06.321316 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 926/1834] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 01:23:05 2022] DELETE /identity/v3/users/07e6e939c82c4864806c0d4df4e93a86 => generated 0 bytes in 502 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:23:06.889980 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb94a16-ad33-4025-9d76-48b55ff7532f admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:06.889980 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-6cb94a16-ad33-4025-9d76-48b55ff7532f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:23:06.906346 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb94a16-ad33-4025-9d76-48b55ff7532f admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:06.962560 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2471a366-2556-4703-ab1d-c0c20f9f2e7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:06.963201 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2471a366-2556-4703-ab1d-c0c20f9f2e7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:06.963485 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2471a366-2556-4703-ab1d-c0c20f9f2e7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:06.963757 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2471a366-2556-4703-ab1d-c0c20f9f2e7f admin admin] PATH_INFO: `/v3/projects/b34af39c52534239b864423d6236bbba` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:06.964905 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2471a366-2556-4703-ab1d-c0c20f9f2e7f admin admin] RBAC: Authorizing `identity:delete_project(project_id=b34af39c52534239b864423d6236bbba)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:06.966829 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2471a366-2556-4703-ab1d-c0c20f9f2e7f admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:07.117813 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-2471a366-2556-4703-ab1d-c0c20f9f2e7f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b34af39c52534239b864423d6236bbba', 'request_id': 'req-2471a366-2556-4703-ab1d-c0c20f9f2e7f'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:07.124169 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-2471a366-2556-4703-ab1d-c0c20f9f2e7f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b34af39c52534239b864423d6236bbba', 'request_id': 'req-2471a366-2556-4703-ab1d-c0c20f9f2e7f'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:07.135478 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-2471a366-2556-4703-ab1d-c0c20f9f2e7f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b34af39c52534239b864423d6236bbba', 'request_id': 'req-2471a366-2556-4703-ab1d-c0c20f9f2e7f'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:07.266393 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-2471a366-2556-4703-ab1d-c0c20f9f2e7f admin admin] The token cache is being invalidate because project b34af39c52534239b864423d6236bbba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81749) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 01:23:07.266933 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-2471a366-2556-4703-ab1d-c0c20f9f2e7f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:07.271936 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-2471a366-2556-4703-ab1d-c0c20f9f2e7f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:07.271936 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 909/1835] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 01:23:06 2022] DELETE /identity/v3/projects/b34af39c52534239b864423d6236bbba => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:23:07.275447 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd09cb1e-1cf6-47dc-8d22-0502b483af34 None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:07.275905 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd09cb1e-1cf6-47dc-8d22-0502b483af34 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:07.276034 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd09cb1e-1cf6-47dc-8d22-0502b483af34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:07.296267 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-dd09cb1e-1cf6-47dc-8d22-0502b483af34 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:23:07.297712 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-dd09cb1e-1cf6-47dc-8d22-0502b483af34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:23:07.366925 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 927/1836] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:23:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:23:07.375624 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75065e62-55e6-4686-bb06-8a21d951c7d1 None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:07.375928 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75065e62-55e6-4686-bb06-8a21d951c7d1 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:07.376206 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75065e62-55e6-4686-bb06-8a21d951c7d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:07.397390 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-75065e62-55e6-4686-bb06-8a21d951c7d1 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:23:07.398643 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-75065e62-55e6-4686-bb06-8a21d951c7d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:23:07.406721 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 910/1837] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:23:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:23:07.411085 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd09cb1e-1cf6-47dc-8d22-0502b483af34 None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:07.421957 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd09cb1e-1cf6-47dc-8d22-0502b483af34 None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:07.426969 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b11fc7-dc30-47bb-b876-a45519736ba1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:07.427576 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b11fc7-dc30-47bb-b876-a45519736ba1 admin admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:07.428434 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b11fc7-dc30-47bb-b876-a45519736ba1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:07.428708 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b11fc7-dc30-47bb-b876-a45519736ba1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:07.429378 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b11fc7-dc30-47bb-b876-a45519736ba1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:23:07.429681 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b11fc7-dc30-47bb-b876-a45519736ba1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:07.431323 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b11fc7-dc30-47bb-b876-a45519736ba1 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:07.445106 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 928/1838] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Aug 31 01:23:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:23:07.449691 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75065e62-55e6-4686-bb06-8a21d951c7d1 None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:07.459711 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75065e62-55e6-4686-bb06-8a21d951c7d1 None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:07.464952 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787347ed-5d52-4c88-9fac-7bd433b51205 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:07.465653 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787347ed-5d52-4c88-9fac-7bd433b51205 admin admin] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:07.466160 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787347ed-5d52-4c88-9fac-7bd433b51205 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:07.466444 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787347ed-5d52-4c88-9fac-7bd433b51205 admin admin] PATH_INFO: `/v3/projects` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:07.467157 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787347ed-5d52-4c88-9fac-7bd433b51205 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:07.469220 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787347ed-5d52-4c88-9fac-7bd433b51205 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:07.577650 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 911/1839] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 01:23:07 2022] POST /identity/v3/projects => generated 399 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:23:07.582423 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b11fc7-dc30-47bb-b876-a45519736ba1 admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:07.593304 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b11fc7-dc30-47bb-b876-a45519736ba1 admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:07.660278 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5383456e-d2aa-467b-9986-a1f126b10bbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:07.661219 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5383456e-d2aa-467b-9986-a1f126b10bbb admin admin] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:07.661668 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5383456e-d2aa-467b-9986-a1f126b10bbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:07.662640 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5383456e-d2aa-467b-9986-a1f126b10bbb admin admin] PATH_INFO: `/v3/users` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:07.663703 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5383456e-d2aa-467b-9986-a1f126b10bbb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:07.665815 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5383456e-d2aa-467b-9986-a1f126b10bbb admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:07.784208 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 929/1840] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 01:23:07 2022] POST /identity/v3/users => generated 300 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:23:07.788849 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787347ed-5d52-4c88-9fac-7bd433b51205 admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:07.800620 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787347ed-5d52-4c88-9fac-7bd433b51205 admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:07.805798 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a8baa6-9578-44c5-ab0a-fa476953ab93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:07.806418 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a8baa6-9578-44c5-ab0a-fa476953ab93 admin admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:07.808521 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a8baa6-9578-44c5-ab0a-fa476953ab93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:07.808521 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a8baa6-9578-44c5-ab0a-fa476953ab93 admin admin] PATH_INFO: `/v3/roles` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:07.808521 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a8baa6-9578-44c5-ab0a-fa476953ab93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:07.809750 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a8baa6-9578-44c5-ab0a-fa476953ab93 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:07.820253 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 912/1841] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:23:07 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:23:07.824530 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5383456e-d2aa-467b-9986-a1f126b10bbb admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:07.835685 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5383456e-d2aa-467b-9986-a1f126b10bbb admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:07.840519 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0729c754-3938-44ac-b956-3908cecde600 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:07.841187 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0729c754-3938-44ac-b956-3908cecde600 admin admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:07.841473 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0729c754-3938-44ac-b956-3908cecde600 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:07.841754 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0729c754-3938-44ac-b956-3908cecde600 admin admin] PATH_INFO: `/v3/roles` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:07.842400 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0729c754-3938-44ac-b956-3908cecde600 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:07.844012 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0729c754-3938-44ac-b956-3908cecde600 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:07.860975 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 930/1842] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:23:07 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 01:23:07.862138 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a8baa6-9578-44c5-ab0a-fa476953ab93 admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:07.884098 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a8baa6-9578-44c5-ab0a-fa476953ab93 admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:07.901960 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced2e290-08d9-4c67-8b98-6b4e54f48a43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:07.902699 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced2e290-08d9-4c67-8b98-6b4e54f48a43 admin admin] REQUEST_METHOD: `PUT` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:07.903042 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced2e290-08d9-4c67-8b98-6b4e54f48a43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:07.903499 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced2e290-08d9-4c67-8b98-6b4e54f48a43 admin admin] PATH_INFO: `/v3/projects/690897ebec3e45c9abe97ba6f73b2b65/users/d0a8658f389d4367a1a52cfb09065835/roles/d0b9768f4d014766b9326387f2e11f34` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:07.919789 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced2e290-08d9-4c67-8b98-6b4e54f48a43 admin admin] RBAC: Authorizing `identity:create_grant(project_id=690897ebec3e45c9abe97ba6f73b2b65, user_id=d0a8658f389d4367a1a52cfb09065835, role_id=d0b9768f4d014766b9326387f2e11f34)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:07.923305 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced2e290-08d9-4c67-8b98-6b4e54f48a43 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:08.026874 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 913/1843] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 01:23:07 2022] PUT /identity/v3/projects/690897ebec3e45c9abe97ba6f73b2b65/users/d0a8658f389d4367a1a52cfb09065835/roles/d0b9768f4d014766b9326387f2e11f34 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:23:08.039771 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6738df7b-c4d6-4903-8e99-ca09a4e08808 None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:08.040065 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6738df7b-c4d6-4903-8e99-ca09a4e08808 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:08.040345 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6738df7b-c4d6-4903-8e99-ca09a4e08808 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:08.068904 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-6738df7b-c4d6-4903-8e99-ca09a4e08808 None None] MFA Rules not processed for user `d0a8658f389d4367a1a52cfb09065835`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:23:08.070328 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-6738df7b-c4d6-4903-8e99-ca09a4e08808 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:23:08.141920 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 931/1844] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:23:08 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:23:08.147127 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced2e290-08d9-4c67-8b98-6b4e54f48a43 admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:08.208858 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced2e290-08d9-4c67-8b98-6b4e54f48a43 admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:08.214311 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae92da07-7097-4668-bf41-3a5f3387f1c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:08.214928 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae92da07-7097-4668-bf41-3a5f3387f1c7 admin admin] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:08.215209 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae92da07-7097-4668-bf41-3a5f3387f1c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:08.215484 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae92da07-7097-4668-bf41-3a5f3387f1c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:08.216123 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae92da07-7097-4668-bf41-3a5f3387f1c7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:08.218079 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae92da07-7097-4668-bf41-3a5f3387f1c7 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:08.316386 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=95,keystone:DELETE=3,keystone:INSERT=2 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:23:08.352807 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 914/1845] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 01:23:08 2022] POST /identity/v3/projects => generated 399 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:23:08.356897 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6738df7b-c4d6-4903-8e99-ca09a4e08808 None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:08.390802 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6738df7b-c4d6-4903-8e99-ca09a4e08808 None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:08.401625 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae92da07-7097-4668-bf41-3a5f3387f1c7 admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:08.403698 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-ae92da07-7097-4668-bf41-3a5f3387f1c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:23:08.440935 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:23:08.524416 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14bea2d-4c0a-4faf-8924-0ccf89e19d4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:08.525085 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14bea2d-4c0a-4faf-8924-0ccf89e19d4c admin admin] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:08.525371 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14bea2d-4c0a-4faf-8924-0ccf89e19d4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:08.525649 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14bea2d-4c0a-4faf-8924-0ccf89e19d4c admin admin] PATH_INFO: `/v3/users` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:08.526345 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae92da07-7097-4668-bf41-3a5f3387f1c7 admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:08.526457 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14bea2d-4c0a-4faf-8924-0ccf89e19d4c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:08.528298 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14bea2d-4c0a-4faf-8924-0ccf89e19d4c admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:08.532879 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d082163-3ff6-4474-81bc-a50559788798 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:08.533610 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d082163-3ff6-4474-81bc-a50559788798 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:08.533962 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d082163-3ff6-4474-81bc-a50559788798 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:08.534311 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d082163-3ff6-4474-81bc-a50559788798 admin admin] PATH_INFO: `/v3/users/68ad33b4f8054e5b9fc4b04c472d8fc6` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:08.535838 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d082163-3ff6-4474-81bc-a50559788798 admin admin] RBAC: Authorizing `identity:delete_user(user_id=68ad33b4f8054e5b9fc4b04c472d8fc6)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:08.538857 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d082163-3ff6-4474-81bc-a50559788798 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:08.805463 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 932/1846] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 01:23:08 2022] POST /identity/v3/users => generated 306 bytes in 449 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 01:23:08.806510 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14bea2d-4c0a-4faf-8924-0ccf89e19d4c admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:08.818936 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14bea2d-4c0a-4faf-8924-0ccf89e19d4c admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:08.824371 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b352e96-a252-4b2b-a698-be172fec84b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:08.825093 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b352e96-a252-4b2b-a698-be172fec84b9 admin admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:08.825453 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b352e96-a252-4b2b-a698-be172fec84b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:08.825973 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b352e96-a252-4b2b-a698-be172fec84b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:08.826691 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b352e96-a252-4b2b-a698-be172fec84b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:08.828318 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b352e96-a252-4b2b-a698-be172fec84b9 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:08.839570 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 933/1847] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:23:08 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:23:08.846496 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b352e96-a252-4b2b-a698-be172fec84b9 admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:08.856963 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b352e96-a252-4b2b-a698-be172fec84b9 admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:08.861562 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f773163f-4ae6-44f0-bc7a-8bb3738d1c4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:08.862210 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f773163f-4ae6-44f0-bc7a-8bb3738d1c4a admin admin] REQUEST_METHOD: `PUT` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:08.862497 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f773163f-4ae6-44f0-bc7a-8bb3738d1c4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:08.862778 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f773163f-4ae6-44f0-bc7a-8bb3738d1c4a admin admin] PATH_INFO: `/v3/projects/fa5bbb1ad028460cbc060d0ec7065efd/users/c04cfb38de834094a178a6815def316a/roles/d009411496b8452b88e77ffb8db015a7` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:08.877851 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f773163f-4ae6-44f0-bc7a-8bb3738d1c4a admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa5bbb1ad028460cbc060d0ec7065efd, user_id=c04cfb38de834094a178a6815def316a, role_id=d009411496b8452b88e77ffb8db015a7)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:08.881415 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-9d082163-3ff6-4474-81bc-a50559788798 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '68ad33b4f8054e5b9fc4b04c472d8fc6'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:08.881536 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f773163f-4ae6-44f0-bc7a-8bb3738d1c4a admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:09.058997 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-9d082163-3ff6-4474-81bc-a50559788798 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '68ad33b4f8054e5b9fc4b04c472d8fc6'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:09.060054 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-9d082163-3ff6-4474-81bc-a50559788798 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '68ad33b4f8054e5b9fc4b04c472d8fc6'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:09.066572 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 934/1848] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 01:23:08 2022] PUT /identity/v3/projects/fa5bbb1ad028460cbc060d0ec7065efd/users/c04cfb38de834094a178a6815def316a/roles/d009411496b8452b88e77ffb8db015a7 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 01:23:09.070457 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-9d082163-3ff6-4474-81bc-a50559788798 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '68ad33b4f8054e5b9fc4b04c472d8fc6'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:09.070780 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-9d082163-3ff6-4474-81bc-a50559788798 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '68ad33b4f8054e5b9fc4b04c472d8fc6'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:09.078147 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f773163f-4ae6-44f0-bc7a-8bb3738d1c4a admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:09.081553 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-f773163f-4ae6-44f0-bc7a-8bb3738d1c4a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:23:09.108621 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 915/1849] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 01:23:08 2022] DELETE /identity/v3/users/68ad33b4f8054e5b9fc4b04c472d8fc6 => generated 0 bytes in 709 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:23:09.112868 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46359fa9-7229-4118-819b-101baebaf06e None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:09.113361 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46359fa9-7229-4118-819b-101baebaf06e None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:09.113789 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46359fa9-7229-4118-819b-101baebaf06e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:09.138724 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-46359fa9-7229-4118-819b-101baebaf06e None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:23:09.141050 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-46359fa9-7229-4118-819b-101baebaf06e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:23:09.174687 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f773163f-4ae6-44f0-bc7a-8bb3738d1c4a admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:09.200317 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 916/1850] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:23:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:23:09.211123 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46359fa9-7229-4118-819b-101baebaf06e None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:09.212477 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-46359fa9-7229-4118-819b-101baebaf06e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:23:09.236253 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8832cc7d-59fc-490f-91a9-9b0cac1ea8eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:09.237377 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8832cc7d-59fc-490f-91a9-9b0cac1ea8eb admin admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:09.237771 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8832cc7d-59fc-490f-91a9-9b0cac1ea8eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:09.238113 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8832cc7d-59fc-490f-91a9-9b0cac1ea8eb admin admin] PATH_INFO: `/v3/roles` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:09.239034 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8832cc7d-59fc-490f-91a9-9b0cac1ea8eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:09.246705 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8832cc7d-59fc-490f-91a9-9b0cac1ea8eb admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:09.253397 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 935/1851] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:23:09 2022] GET /identity/v3/roles => generated 1420 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:23:09.257697 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8832cc7d-59fc-490f-91a9-9b0cac1ea8eb admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:09.268378 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8832cc7d-59fc-490f-91a9-9b0cac1ea8eb admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:09.273733 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdec0b49-9480-4b19-a65f-281212ebfab5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:09.274402 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdec0b49-9480-4b19-a65f-281212ebfab5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:09.274687 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdec0b49-9480-4b19-a65f-281212ebfab5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:09.275093 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdec0b49-9480-4b19-a65f-281212ebfab5 admin admin] PATH_INFO: `/v3/projects/fa5bbb1ad028460cbc060d0ec7065efd/users/c04cfb38de834094a178a6815def316a/roles/d009411496b8452b88e77ffb8db015a7` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:09.277233 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdec0b49-9480-4b19-a65f-281212ebfab5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa5bbb1ad028460cbc060d0ec7065efd, user_id=c04cfb38de834094a178a6815def316a, role_id=d009411496b8452b88e77ffb8db015a7)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:09.279890 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdec0b49-9480-4b19-a65f-281212ebfab5 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:09.284392 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46359fa9-7229-4118-819b-101baebaf06e None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:09.290499 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 936/1852] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 01:23:09 2022] PUT /identity/v3/projects/fa5bbb1ad028460cbc060d0ec7065efd/users/c04cfb38de834094a178a6815def316a/roles/d009411496b8452b88e77ffb8db015a7 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 01:23:09.300513 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f595c2-8662-4fef-9de2-67a5701f62f7 None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:09.300922 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f595c2-8662-4fef-9de2-67a5701f62f7 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:09.301203 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f595c2-8662-4fef-9de2-67a5701f62f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:09.331311 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-a8f595c2-8662-4fef-9de2-67a5701f62f7 None None] MFA Rules not processed for user `c04cfb38de834094a178a6815def316a`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:23:09.332517 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-a8f595c2-8662-4fef-9de2-67a5701f62f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:23:09.337157 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c276fc-51df-4a41-b8ab-fcc16b64f0f9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0817c3320d5d41d8bba0550c8ae8dd60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:09.338002 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c276fc-51df-4a41-b8ab-fcc16b64f0f9 service neutron] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:09.338288 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c276fc-51df-4a41-b8ab-fcc16b64f0f9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:09.338707 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c276fc-51df-4a41-b8ab-fcc16b64f0f9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:09.415587 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c276fc-51df-4a41-b8ab-fcc16b64f0f9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:09.417100 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c276fc-51df-4a41-b8ab-fcc16b64f0f9 service neutron] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:09.428697 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 917/1853] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:23:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 01:23:09.429830 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 937/1854] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:23:09 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 01:23:09.442342 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c276fc-51df-4a41-b8ab-fcc16b64f0f9 service neutron] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:09.449520 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-d8c276fc-51df-4a41-b8ab-fcc16b64f0f9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:23:09.519820 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c276fc-51df-4a41-b8ab-fcc16b64f0f9 service neutron] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:09.523955 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d9be5c-2b4c-4ce9-a30e-41460dd82535 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5c8fc69c518412ba1aad61a9f4f558c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:09.524616 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d9be5c-2b4c-4ce9-a30e-41460dd82535 service cinder] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:09.524933 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d9be5c-2b4c-4ce9-a30e-41460dd82535 service cinder] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:09.525205 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d9be5c-2b4c-4ce9-a30e-41460dd82535 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:09.536776 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d9be5c-2b4c-4ce9-a30e-41460dd82535 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:09.538299 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d9be5c-2b4c-4ce9-a30e-41460dd82535 service cinder] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:09.550024 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 918/1855] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:23:09 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:23:09.993444 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f595c2-8662-4fef-9de2-67a5701f62f7 None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:10.006466 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f595c2-8662-4fef-9de2-67a5701f62f7 None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:10.022335 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf36c947-42df-4154-b720-dcb54102b80f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5c8fc69c518412ba1aad61a9f4f558c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:10.023051 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf36c947-42df-4154-b720-dcb54102b80f service cinder] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:10.025734 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf36c947-42df-4154-b720-dcb54102b80f service cinder] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:10.025734 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf36c947-42df-4154-b720-dcb54102b80f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:10.032141 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-cf36c947-42df-4154-b720-dcb54102b80f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:23:10.173946 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf36c947-42df-4154-b720-dcb54102b80f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:10.176793 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf36c947-42df-4154-b720-dcb54102b80f service cinder] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:10.188502 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 938/1856] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:23:09 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:23:10.291257 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d9be5c-2b4c-4ce9-a30e-41460dd82535 service cinder] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:10.293358 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-25d9be5c-2b4c-4ce9-a30e-41460dd82535 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:23:10.316160 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d9be5c-2b4c-4ce9-a30e-41460dd82535 service cinder] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:10.329530 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78fb608-0505-43a2-9693-b538461a76d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:10.330696 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78fb608-0505-43a2-9693-b538461a76d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:10.331392 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78fb608-0505-43a2-9693-b538461a76d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:10.332445 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78fb608-0505-43a2-9693-b538461a76d8 admin admin] PATH_INFO: `/v3/projects/41c0ed31afdf4211beb6f2640905d764` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:10.335280 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78fb608-0505-43a2-9693-b538461a76d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=41c0ed31afdf4211beb6f2640905d764)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:10.337500 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78fb608-0505-43a2-9693-b538461a76d8 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:10.507926 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-f78fb608-0505-43a2-9693-b538461a76d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '41c0ed31afdf4211beb6f2640905d764', 'request_id': 'req-f78fb608-0505-43a2-9693-b538461a76d8'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:10.513304 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-f78fb608-0505-43a2-9693-b538461a76d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '41c0ed31afdf4211beb6f2640905d764', 'request_id': 'req-f78fb608-0505-43a2-9693-b538461a76d8'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:10.519814 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-f78fb608-0505-43a2-9693-b538461a76d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '41c0ed31afdf4211beb6f2640905d764', 'request_id': 'req-f78fb608-0505-43a2-9693-b538461a76d8'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:10.606807 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-f78fb608-0505-43a2-9693-b538461a76d8 admin admin] The token cache is being invalidate because project 41c0ed31afdf4211beb6f2640905d764 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81749) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 01:23:10.607109 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-f78fb608-0505-43a2-9693-b538461a76d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:10.609130 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-f78fb608-0505-43a2-9693-b538461a76d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:10.610999 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 919/1857] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 01:23:10 2022] DELETE /identity/v3/projects/41c0ed31afdf4211beb6f2640905d764 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:23:10.614814 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf36c947-42df-4154-b720-dcb54102b80f service cinder] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:10.628318 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf36c947-42df-4154-b720-dcb54102b80f service cinder] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:10.717544 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1d8d2f-42e7-41fd-aa55-dc967e8c3d95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:10.718375 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1d8d2f-42e7-41fd-aa55-dc967e8c3d95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:10.718810 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1d8d2f-42e7-41fd-aa55-dc967e8c3d95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:10.719237 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1d8d2f-42e7-41fd-aa55-dc967e8c3d95 admin admin] PATH_INFO: `/v3/users/b131c52388134817a66860e60bc363a7` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:10.726305 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1d8d2f-42e7-41fd-aa55-dc967e8c3d95 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b131c52388134817a66860e60bc363a7)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:10.726305 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1d8d2f-42e7-41fd-aa55-dc967e8c3d95 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:10.965255 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-8e1d8d2f-42e7-41fd-aa55-dc967e8c3d95 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b131c52388134817a66860e60bc363a7'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:10.988426 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-8e1d8d2f-42e7-41fd-aa55-dc967e8c3d95 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b131c52388134817a66860e60bc363a7'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:10.989529 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-8e1d8d2f-42e7-41fd-aa55-dc967e8c3d95 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b131c52388134817a66860e60bc363a7'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:11.001888 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-8e1d8d2f-42e7-41fd-aa55-dc967e8c3d95 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b131c52388134817a66860e60bc363a7'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:11.096717 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-8e1d8d2f-42e7-41fd-aa55-dc967e8c3d95 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b131c52388134817a66860e60bc363a7'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:11.098494 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:23:11.105527 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 939/1858] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 01:23:10 2022] DELETE /identity/v3/users/b131c52388134817a66860e60bc363a7 => generated 0 bytes in 491 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:23:11.549471 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78fb608-0505-43a2-9693-b538461a76d8 admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:11.552397 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-f78fb608-0505-43a2-9693-b538461a76d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:23:11.570548 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78fb608-0505-43a2-9693-b538461a76d8 admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:11.632093 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b2fa4a-63bd-4fc4-ae34-c2f63d637d80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:11.632699 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b2fa4a-63bd-4fc4-ae34-c2f63d637d80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:11.633007 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b2fa4a-63bd-4fc4-ae34-c2f63d637d80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:11.633281 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b2fa4a-63bd-4fc4-ae34-c2f63d637d80 admin admin] PATH_INFO: `/v3/projects/b46c20215e15449daeea6291edeceb59` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:11.634462 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b2fa4a-63bd-4fc4-ae34-c2f63d637d80 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b46c20215e15449daeea6291edeceb59)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:11.636414 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b2fa4a-63bd-4fc4-ae34-c2f63d637d80 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:11.815495 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-55b2fa4a-63bd-4fc4-ae34-c2f63d637d80 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b46c20215e15449daeea6291edeceb59', 'request_id': 'req-55b2fa4a-63bd-4fc4-ae34-c2f63d637d80'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:11.821297 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-55b2fa4a-63bd-4fc4-ae34-c2f63d637d80 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b46c20215e15449daeea6291edeceb59', 'request_id': 'req-55b2fa4a-63bd-4fc4-ae34-c2f63d637d80'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:11.828072 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-55b2fa4a-63bd-4fc4-ae34-c2f63d637d80 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b46c20215e15449daeea6291edeceb59', 'request_id': 'req-55b2fa4a-63bd-4fc4-ae34-c2f63d637d80'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:11.994940 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-55b2fa4a-63bd-4fc4-ae34-c2f63d637d80 admin admin] The token cache is being invalidate because project b46c20215e15449daeea6291edeceb59 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81749) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 01:23:11.994940 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-55b2fa4a-63bd-4fc4-ae34-c2f63d637d80 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:11.994940 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-55b2fa4a-63bd-4fc4-ae34-c2f63d637d80 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:11.996475 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 920/1859] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 01:23:11 2022] DELETE /identity/v3/projects/b46c20215e15449daeea6291edeceb59 => generated 0 bytes in 447 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:23:12.002802 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7663c96-7fa6-4885-8ec9-adacc35fac2b None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:12.003342 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7663c96-7fa6-4885-8ec9-adacc35fac2b None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:12.003552 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7663c96-7fa6-4885-8ec9-adacc35fac2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:12.027920 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-e7663c96-7fa6-4885-8ec9-adacc35fac2b None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:23:12.029270 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-e7663c96-7fa6-4885-8ec9-adacc35fac2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:23:12.090381 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 940/1860] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:23:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:23:12.098800 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740c4437-5a0e-4bf9-826a-1725ec5ba45e None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:12.099093 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740c4437-5a0e-4bf9-826a-1725ec5ba45e None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:12.099409 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740c4437-5a0e-4bf9-826a-1725ec5ba45e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:12.125828 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-740c4437-5a0e-4bf9-826a-1725ec5ba45e None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:23:12.127819 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-740c4437-5a0e-4bf9-826a-1725ec5ba45e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:23:12.142390 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 921/1861] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:23:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:23:12.143617 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7663c96-7fa6-4885-8ec9-adacc35fac2b None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:12.157945 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7663c96-7fa6-4885-8ec9-adacc35fac2b None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:12.164966 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ab5801-4c37-4779-a4fd-11db49bc1480 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:12.165808 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ab5801-4c37-4779-a4fd-11db49bc1480 admin admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:12.166264 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ab5801-4c37-4779-a4fd-11db49bc1480 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:12.166694 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ab5801-4c37-4779-a4fd-11db49bc1480 admin admin] PATH_INFO: `/v3/domains` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:12.167388 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ab5801-4c37-4779-a4fd-11db49bc1480 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:23:12.167763 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ab5801-4c37-4779-a4fd-11db49bc1480 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:12.169418 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ab5801-4c37-4779-a4fd-11db49bc1480 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:12.183500 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 941/1862] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Aug 31 01:23:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:23:12.191979 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740c4437-5a0e-4bf9-826a-1725ec5ba45e None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:12.202051 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740c4437-5a0e-4bf9-826a-1725ec5ba45e None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:12.207213 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6f5376-266c-485c-abf7-b847d8dcc772 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:12.207956 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6f5376-266c-485c-abf7-b847d8dcc772 admin admin] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:12.208360 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6f5376-266c-485c-abf7-b847d8dcc772 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:12.208814 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6f5376-266c-485c-abf7-b847d8dcc772 admin admin] PATH_INFO: `/v3/projects` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:12.210052 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6f5376-266c-485c-abf7-b847d8dcc772 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:12.212098 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6f5376-266c-485c-abf7-b847d8dcc772 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:12.322939 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:23:12.327933 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 922/1863] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 01:23:12 2022] POST /identity/v3/projects => generated 385 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:23:12.334875 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ab5801-4c37-4779-a4fd-11db49bc1480 admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:12.347352 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ab5801-4c37-4779-a4fd-11db49bc1480 admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:12.409609 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03e6d7a-c175-4287-adf0-daad2af03a44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:12.410348 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03e6d7a-c175-4287-adf0-daad2af03a44 admin admin] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:12.410709 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03e6d7a-c175-4287-adf0-daad2af03a44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:12.410987 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03e6d7a-c175-4287-adf0-daad2af03a44 admin admin] PATH_INFO: `/v3/users` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:12.411620 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03e6d7a-c175-4287-adf0-daad2af03a44 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:12.413596 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03e6d7a-c175-4287-adf0-daad2af03a44 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:12.558342 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 942/1864] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 01:23:12 2022] POST /identity/v3/users => generated 293 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:23:12.562557 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6f5376-266c-485c-abf7-b847d8dcc772 admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:12.573713 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6f5376-266c-485c-abf7-b847d8dcc772 admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:12.578386 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bc6b10-69cd-408e-b550-e59b7ad729b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:12.579325 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bc6b10-69cd-408e-b550-e59b7ad729b4 admin admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:12.579980 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bc6b10-69cd-408e-b550-e59b7ad729b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:12.580418 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bc6b10-69cd-408e-b550-e59b7ad729b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:12.584847 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bc6b10-69cd-408e-b550-e59b7ad729b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:12.584847 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bc6b10-69cd-408e-b550-e59b7ad729b4 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:12.594058 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 923/1865] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:23:12 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:23:12.598123 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03e6d7a-c175-4287-adf0-daad2af03a44 admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:12.608644 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03e6d7a-c175-4287-adf0-daad2af03a44 admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:12.613726 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef5f5ba-7b44-410b-8c4e-149cddc5115a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:12.614338 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef5f5ba-7b44-410b-8c4e-149cddc5115a admin admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:12.614618 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef5f5ba-7b44-410b-8c4e-149cddc5115a admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:12.615372 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef5f5ba-7b44-410b-8c4e-149cddc5115a admin admin] PATH_INFO: `/v3/roles` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:12.616015 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef5f5ba-7b44-410b-8c4e-149cddc5115a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:12.617646 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef5f5ba-7b44-410b-8c4e-149cddc5115a admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:12.628743 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 943/1866] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:23:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 01:23:12.633564 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bc6b10-69cd-408e-b550-e59b7ad729b4 admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:12.643431 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bc6b10-69cd-408e-b550-e59b7ad729b4 admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:12.647944 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3300f1aa-a2e1-4ab3-a660-35f173684e0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:12.648588 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3300f1aa-a2e1-4ab3-a660-35f173684e0a admin admin] REQUEST_METHOD: `PUT` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:12.648906 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3300f1aa-a2e1-4ab3-a660-35f173684e0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:12.649183 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3300f1aa-a2e1-4ab3-a660-35f173684e0a admin admin] PATH_INFO: `/v3/projects/847666a3a4ea46e1a3a5180ea2625a92/users/bfee7e29a52b4da6bdbff237197bbd30/roles/d0b9768f4d014766b9326387f2e11f34` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:12.666817 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3300f1aa-a2e1-4ab3-a660-35f173684e0a admin admin] RBAC: Authorizing `identity:create_grant(project_id=847666a3a4ea46e1a3a5180ea2625a92, user_id=bfee7e29a52b4da6bdbff237197bbd30, role_id=d0b9768f4d014766b9326387f2e11f34)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:12.667018 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3300f1aa-a2e1-4ab3-a660-35f173684e0a admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:12.781094 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 924/1867] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 01:23:12 2022] PUT /identity/v3/projects/847666a3a4ea46e1a3a5180ea2625a92/users/bfee7e29a52b4da6bdbff237197bbd30/roles/d0b9768f4d014766b9326387f2e11f34 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:23:12.791884 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740e2e37-0133-4c57-b65e-3f9d13d704f6 None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:12.792173 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740e2e37-0133-4c57-b65e-3f9d13d704f6 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:12.792449 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740e2e37-0133-4c57-b65e-3f9d13d704f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:12.823131 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-740e2e37-0133-4c57-b65e-3f9d13d704f6 None None] MFA Rules not processed for user `bfee7e29a52b4da6bdbff237197bbd30`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:23:12.824439 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-740e2e37-0133-4c57-b65e-3f9d13d704f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:23:12.886576 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 944/1868] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:23:12 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 01:23:12.967530 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3300f1aa-a2e1-4ab3-a660-35f173684e0a admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:12.969922 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-3300f1aa-a2e1-4ab3-a660-35f173684e0a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:23:13.057718 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3300f1aa-a2e1-4ab3-a660-35f173684e0a admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:13.070560 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c38eef-dd2e-4555-b29b-36778496ff5a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5c8fc69c518412ba1aad61a9f4f558c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:13.071213 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c38eef-dd2e-4555-b29b-36778496ff5a service cinder] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:13.071512 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c38eef-dd2e-4555-b29b-36778496ff5a service cinder] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:13.071772 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c38eef-dd2e-4555-b29b-36778496ff5a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:13.081001 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c38eef-dd2e-4555-b29b-36778496ff5a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:13.082437 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c38eef-dd2e-4555-b29b-36778496ff5a service cinder] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:13.092398 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 925/1869] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:23:12 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:23:13.134668 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740e2e37-0133-4c57-b65e-3f9d13d704f6 None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:13.198017 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740e2e37-0133-4c57-b65e-3f9d13d704f6 None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:13.205505 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646b88ac-18f3-4a1e-bbe3-b4ac87cee61b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:13.206023 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646b88ac-18f3-4a1e-bbe3-b4ac87cee61b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:13.206306 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646b88ac-18f3-4a1e-bbe3-b4ac87cee61b admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:13.206586 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646b88ac-18f3-4a1e-bbe3-b4ac87cee61b admin admin] PATH_INFO: `/v3/users/bfee7e29a52b4da6bdbff237197bbd30` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:13.207768 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646b88ac-18f3-4a1e-bbe3-b4ac87cee61b admin admin] RBAC: Authorizing `identity:delete_user(user_id=bfee7e29a52b4da6bdbff237197bbd30)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:13.211839 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646b88ac-18f3-4a1e-bbe3-b4ac87cee61b admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:13.354414 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c38eef-dd2e-4555-b29b-36778496ff5a service cinder] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:13.355819 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-47c38eef-dd2e-4555-b29b-36778496ff5a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:23:13.369508 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c38eef-dd2e-4555-b29b-36778496ff5a service cinder] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:13.374191 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bda5fd-5c91-44fa-88c7-f5dfba2b9a18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:13.374814 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bda5fd-5c91-44fa-88c7-f5dfba2b9a18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:13.376244 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bda5fd-5c91-44fa-88c7-f5dfba2b9a18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:13.376244 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bda5fd-5c91-44fa-88c7-f5dfba2b9a18 admin admin] PATH_INFO: `/v3/users/af0c6ebaa47447709fd94b142923b1ee` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:13.376457 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bda5fd-5c91-44fa-88c7-f5dfba2b9a18 admin admin] RBAC: Authorizing `identity:delete_user(user_id=af0c6ebaa47447709fd94b142923b1ee)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:13.378389 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bda5fd-5c91-44fa-88c7-f5dfba2b9a18 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:13.410020 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-646b88ac-18f3-4a1e-bbe3-b4ac87cee61b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bfee7e29a52b4da6bdbff237197bbd30'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:13.421869 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-646b88ac-18f3-4a1e-bbe3-b4ac87cee61b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bfee7e29a52b4da6bdbff237197bbd30'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:13.422575 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-646b88ac-18f3-4a1e-bbe3-b4ac87cee61b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bfee7e29a52b4da6bdbff237197bbd30'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:13.429449 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-646b88ac-18f3-4a1e-bbe3-b4ac87cee61b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bfee7e29a52b4da6bdbff237197bbd30'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:13.506805 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-646b88ac-18f3-4a1e-bbe3-b4ac87cee61b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bfee7e29a52b4da6bdbff237197bbd30'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:13.513466 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 945/1870] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 01:23:13 2022] DELETE /identity/v3/users/bfee7e29a52b4da6bdbff237197bbd30 => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:23:13.517881 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69afb99-e666-4d22-98af-063fb8df2101 None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:13.518175 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69afb99-e666-4d22-98af-063fb8df2101 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:13.518990 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69afb99-e666-4d22-98af-063fb8df2101 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:13.537866 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-a69afb99-e666-4d22-98af-063fb8df2101 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:23:13.539111 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-a69afb99-e666-4d22-98af-063fb8df2101 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:23:13.587361 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-63bda5fd-5c91-44fa-88c7-f5dfba2b9a18 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af0c6ebaa47447709fd94b142923b1ee'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:13.599035 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 946/1871] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:23:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:23:13.608880 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69afb99-e666-4d22-98af-063fb8df2101 None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:13.610236 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-a69afb99-e666-4d22-98af-063fb8df2101 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:23:13.660383 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-63bda5fd-5c91-44fa-88c7-f5dfba2b9a18 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af0c6ebaa47447709fd94b142923b1ee'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:13.661040 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-63bda5fd-5c91-44fa-88c7-f5dfba2b9a18 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af0c6ebaa47447709fd94b142923b1ee'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:13.665186 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69afb99-e666-4d22-98af-063fb8df2101 None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:13.668579 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-63bda5fd-5c91-44fa-88c7-f5dfba2b9a18 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af0c6ebaa47447709fd94b142923b1ee'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:13.668699 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8355ecca-e215-4378-871a-3c80a488a763 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0817c3320d5d41d8bba0550c8ae8dd60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:13.669240 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-63bda5fd-5c91-44fa-88c7-f5dfba2b9a18 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af0c6ebaa47447709fd94b142923b1ee'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:13.669352 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8355ecca-e215-4378-871a-3c80a488a763 service neutron] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:13.669564 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8355ecca-e215-4378-871a-3c80a488a763 service neutron] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:13.669836 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8355ecca-e215-4378-871a-3c80a488a763 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:13.671505 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-8355ecca-e215-4378-871a-3c80a488a763 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:23:13.683623 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8355ecca-e215-4378-871a-3c80a488a763 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:13.685858 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8355ecca-e215-4378-871a-3c80a488a763 service neutron] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:13.688266 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 926/1872] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 01:23:13 2022] DELETE /identity/v3/users/af0c6ebaa47447709fd94b142923b1ee => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:23:13.692887 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfb966f-4c78-42c0-9909-093c892aabfc None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:13.694204 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfb966f-4c78-42c0-9909-093c892aabfc None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:13.694717 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfb966f-4c78-42c0-9909-093c892aabfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:13.697697 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 947/1873] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:23:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 01:23:13.723482 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-fcfb966f-4c78-42c0-9909-093c892aabfc None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:23:13.724878 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-fcfb966f-4c78-42c0-9909-093c892aabfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:23:13.730960 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:23:13.799243 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 927/1874] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:23:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 01:23:13.806135 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8355ecca-e215-4378-871a-3c80a488a763 service neutron] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:13.807628 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-8355ecca-e215-4378-871a-3c80a488a763 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:23:13.819323 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8355ecca-e215-4378-871a-3c80a488a763 service neutron] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:13.867832 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6cb6a0-2997-4351-abb3-f7884d61d6a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0817c3320d5d41d8bba0550c8ae8dd60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:13.869011 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6cb6a0-2997-4351-abb3-f7884d61d6a5 service neutron] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:13.869130 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6cb6a0-2997-4351-abb3-f7884d61d6a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:13.869336 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6cb6a0-2997-4351-abb3-f7884d61d6a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:13.879576 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6cb6a0-2997-4351-abb3-f7884d61d6a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:13.882024 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6cb6a0-2997-4351-abb3-f7884d61d6a5 service neutron] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:13.892669 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 948/1875] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:23:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:23:14.110329 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfb966f-4c78-42c0-9909-093c892aabfc None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:14.112489 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-fcfb966f-4c78-42c0-9909-093c892aabfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:23:14.134808 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfb966f-4c78-42c0-9909-093c892aabfc None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:14.173085 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6a6415-8c83-484e-881e-0e8daa44c436 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:14.173085 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6a6415-8c83-484e-881e-0e8daa44c436 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:14.173085 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6a6415-8c83-484e-881e-0e8daa44c436 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:14.173085 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6a6415-8c83-484e-881e-0e8daa44c436 admin admin] PATH_INFO: `/v3/projects/847666a3a4ea46e1a3a5180ea2625a92` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:14.176961 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6a6415-8c83-484e-881e-0e8daa44c436 admin admin] RBAC: Authorizing `identity:delete_project(project_id=847666a3a4ea46e1a3a5180ea2625a92)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:14.179491 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6a6415-8c83-484e-881e-0e8daa44c436 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:14.360560 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-4c6a6415-8c83-484e-881e-0e8daa44c436 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '847666a3a4ea46e1a3a5180ea2625a92', 'request_id': 'req-4c6a6415-8c83-484e-881e-0e8daa44c436'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:14.368046 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-4c6a6415-8c83-484e-881e-0e8daa44c436 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '847666a3a4ea46e1a3a5180ea2625a92', 'request_id': 'req-4c6a6415-8c83-484e-881e-0e8daa44c436'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:14.378014 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-4c6a6415-8c83-484e-881e-0e8daa44c436 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '847666a3a4ea46e1a3a5180ea2625a92', 'request_id': 'req-4c6a6415-8c83-484e-881e-0e8daa44c436'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:14.484221 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:23:14.491916 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6cb6a0-2997-4351-abb3-f7884d61d6a5 service neutron] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:14.493576 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-2e6cb6a0-2997-4351-abb3-f7884d61d6a5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:23:14.516473 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-4c6a6415-8c83-484e-881e-0e8daa44c436 admin admin] The token cache is being invalidate because project 847666a3a4ea46e1a3a5180ea2625a92 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81749) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 01:23:14.516827 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-4c6a6415-8c83-484e-881e-0e8daa44c436 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:14.517524 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-4c6a6415-8c83-484e-881e-0e8daa44c436 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:14.519719 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 928/1876] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 01:23:14 2022] DELETE /identity/v3/projects/847666a3a4ea46e1a3a5180ea2625a92 => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:23:14.521614 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6cb6a0-2997-4351-abb3-f7884d61d6a5 service neutron] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:14.525756 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c328f18e-a356-4a14-a3e7-c9e9130fb035 None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:14.526695 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c328f18e-a356-4a14-a3e7-c9e9130fb035 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:14.527426 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c328f18e-a356-4a14-a3e7-c9e9130fb035 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:14.562462 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-c328f18e-a356-4a14-a3e7-c9e9130fb035 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:23:14.564036 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-c328f18e-a356-4a14-a3e7-c9e9130fb035 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:23:14.620225 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac29b13-5717-497d-9dbe-d94dc8ec10ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:14.620910 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac29b13-5717-497d-9dbe-d94dc8ec10ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:14.621187 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac29b13-5717-497d-9dbe-d94dc8ec10ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:14.621461 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac29b13-5717-497d-9dbe-d94dc8ec10ce admin admin] PATH_INFO: `/v3/projects/52b8c1fb7247417b9440c0a4738a5a5b` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:14.623302 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac29b13-5717-497d-9dbe-d94dc8ec10ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=52b8c1fb7247417b9440c0a4738a5a5b)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:14.627121 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac29b13-5717-497d-9dbe-d94dc8ec10ce admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:14.671655 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 929/1877] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:23:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:23:14.681120 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb727421-6bc1-49e3-95bb-6252a76210b1 None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:14.681415 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb727421-6bc1-49e3-95bb-6252a76210b1 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:14.681706 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb727421-6bc1-49e3-95bb-6252a76210b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:14.703345 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-fb727421-6bc1-49e3-95bb-6252a76210b1 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:23:14.704984 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-fb727421-6bc1-49e3-95bb-6252a76210b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:23:14.713413 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 930/1878] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:23:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 01:23:14.717763 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb727421-6bc1-49e3-95bb-6252a76210b1 None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:14.727259 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb727421-6bc1-49e3-95bb-6252a76210b1 None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:14.731963 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2687ae84-6469-4aed-9b73-32e693e1abca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:14.732593 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2687ae84-6469-4aed-9b73-32e693e1abca admin admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:14.732955 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2687ae84-6469-4aed-9b73-32e693e1abca admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:14.733222 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2687ae84-6469-4aed-9b73-32e693e1abca admin admin] PATH_INFO: `/v3/domains` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:14.733874 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2687ae84-6469-4aed-9b73-32e693e1abca admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:23:14.734187 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2687ae84-6469-4aed-9b73-32e693e1abca admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:14.735812 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2687ae84-6469-4aed-9b73-32e693e1abca admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:14.751286 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 931/1879] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Aug 31 01:23:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:23:14.752343 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2687ae84-6469-4aed-9b73-32e693e1abca admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:14.767732 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2687ae84-6469-4aed-9b73-32e693e1abca admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:14.779247 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-4ac29b13-5717-497d-9dbe-d94dc8ec10ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '52b8c1fb7247417b9440c0a4738a5a5b', 'request_id': 'req-4ac29b13-5717-497d-9dbe-d94dc8ec10ce'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:14.786078 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-4ac29b13-5717-497d-9dbe-d94dc8ec10ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '52b8c1fb7247417b9440c0a4738a5a5b', 'request_id': 'req-4ac29b13-5717-497d-9dbe-d94dc8ec10ce'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:14.806010 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:23:14.828216 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b145fee-d246-4b48-a549-9ea89f5d0801 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:14.828913 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b145fee-d246-4b48-a549-9ea89f5d0801 admin admin] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:14.829410 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b145fee-d246-4b48-a549-9ea89f5d0801 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:14.829684 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b145fee-d246-4b48-a549-9ea89f5d0801 admin admin] PATH_INFO: `/v3/projects` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:14.830311 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b145fee-d246-4b48-a549-9ea89f5d0801 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:14.832217 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b145fee-d246-4b48-a549-9ea89f5d0801 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:14.922890 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-4ac29b13-5717-497d-9dbe-d94dc8ec10ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '52b8c1fb7247417b9440c0a4738a5a5b', 'request_id': 'req-4ac29b13-5717-497d-9dbe-d94dc8ec10ce'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:14.935361 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-4ac29b13-5717-497d-9dbe-d94dc8ec10ce admin admin] The token cache is being invalidate because project 52b8c1fb7247417b9440c0a4738a5a5b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81750) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 01:23:14.935653 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-4ac29b13-5717-497d-9dbe-d94dc8ec10ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:14.936290 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-4ac29b13-5717-497d-9dbe-d94dc8ec10ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:14.937886 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 949/1880] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 01:23:14 2022] DELETE /identity/v3/projects/52b8c1fb7247417b9440c0a4738a5a5b => generated 0 bytes in 447 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:23:14.938646 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac29b13-5717-497d-9dbe-d94dc8ec10ce admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:14.939906 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-4ac29b13-5717-497d-9dbe-d94dc8ec10ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:23:14.951767 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac29b13-5717-497d-9dbe-d94dc8ec10ce admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:14.956702 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87efc2fe-dbe1-4d82-98f1-0e778a0f03fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:14.957328 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87efc2fe-dbe1-4d82-98f1-0e778a0f03fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:14.957613 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87efc2fe-dbe1-4d82-98f1-0e778a0f03fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:14.957894 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87efc2fe-dbe1-4d82-98f1-0e778a0f03fa admin admin] PATH_INFO: `/v3/users/4f0a1c35f10f4e509b386358d28ef427` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:14.959012 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87efc2fe-dbe1-4d82-98f1-0e778a0f03fa admin admin] RBAC: Authorizing `identity:delete_user(user_id=4f0a1c35f10f4e509b386358d28ef427)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:14.960947 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87efc2fe-dbe1-4d82-98f1-0e778a0f03fa admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:14.985641 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 932/1881] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 01:23:14 2022] POST /identity/v3/projects => generated 371 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:23:14.985641 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b145fee-d246-4b48-a549-9ea89f5d0801 admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:14.985641 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-9b145fee-d246-4b48-a549-9ea89f5d0801 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:23:15.046590 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b145fee-d246-4b48-a549-9ea89f5d0801 admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:15.051292 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef5ee6c-b8de-468c-83f2-92d3e2eda5b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:15.051924 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef5ee6c-b8de-468c-83f2-92d3e2eda5b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:15.052209 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef5ee6c-b8de-468c-83f2-92d3e2eda5b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:15.053175 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef5ee6c-b8de-468c-83f2-92d3e2eda5b6 admin admin] PATH_INFO: `/v3/users/bbeedb790e734326a85a77f5a1b237b2` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:15.054314 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef5ee6c-b8de-468c-83f2-92d3e2eda5b6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bbeedb790e734326a85a77f5a1b237b2)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:15.056250 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef5ee6c-b8de-468c-83f2-92d3e2eda5b6 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:15.153160 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-87efc2fe-dbe1-4d82-98f1-0e778a0f03fa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4f0a1c35f10f4e509b386358d28ef427'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:15.165182 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-87efc2fe-dbe1-4d82-98f1-0e778a0f03fa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4f0a1c35f10f4e509b386358d28ef427'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:15.165963 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-87efc2fe-dbe1-4d82-98f1-0e778a0f03fa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4f0a1c35f10f4e509b386358d28ef427'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:15.172941 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-87efc2fe-dbe1-4d82-98f1-0e778a0f03fa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4f0a1c35f10f4e509b386358d28ef427'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:15.242938 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-87efc2fe-dbe1-4d82-98f1-0e778a0f03fa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4f0a1c35f10f4e509b386358d28ef427'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:15.250535 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 950/1882] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 01:23:14 2022] DELETE /identity/v3/users/4f0a1c35f10f4e509b386358d28ef427 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:23:15.251413 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87efc2fe-dbe1-4d82-98f1-0e778a0f03fa admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:15.252991 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-87efc2fe-dbe1-4d82-98f1-0e778a0f03fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:23:15.261038 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-bef5ee6c-b8de-468c-83f2-92d3e2eda5b6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bbeedb790e734326a85a77f5a1b237b2'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:15.318253 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87efc2fe-dbe1-4d82-98f1-0e778a0f03fa admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:15.322906 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0f30e1-d622-40c1-908a-64ad759e5434 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:15.323514 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0f30e1-d622-40c1-908a-64ad759e5434 admin admin] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:15.323798 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0f30e1-d622-40c1-908a-64ad759e5434 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:15.324074 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0f30e1-d622-40c1-908a-64ad759e5434 admin admin] PATH_INFO: `/v3/users` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:15.324710 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0f30e1-d622-40c1-908a-64ad759e5434 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:15.326785 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0f30e1-d622-40c1-908a-64ad759e5434 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:15.368608 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-bef5ee6c-b8de-468c-83f2-92d3e2eda5b6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bbeedb790e734326a85a77f5a1b237b2'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:15.369197 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-bef5ee6c-b8de-468c-83f2-92d3e2eda5b6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bbeedb790e734326a85a77f5a1b237b2'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:15.376534 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-bef5ee6c-b8de-468c-83f2-92d3e2eda5b6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bbeedb790e734326a85a77f5a1b237b2'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:15.377208 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-bef5ee6c-b8de-468c-83f2-92d3e2eda5b6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bbeedb790e734326a85a77f5a1b237b2'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:15.394478 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 933/1883] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 01:23:14 2022] DELETE /identity/v3/users/bbeedb790e734326a85a77f5a1b237b2 => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:23:15.395831 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383fc6dd-f228-4828-8a1e-a2af9653ad1a None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:15.396647 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383fc6dd-f228-4828-8a1e-a2af9653ad1a None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:15.396956 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383fc6dd-f228-4828-8a1e-a2af9653ad1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:15.419488 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-383fc6dd-f228-4828-8a1e-a2af9653ad1a None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:23:15.420980 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-383fc6dd-f228-4828-8a1e-a2af9653ad1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:23:15.438452 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 951/1884] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 01:23:15 2022] POST /identity/v3/users => generated 286 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:23:15.443793 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0f30e1-d622-40c1-908a-64ad759e5434 admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:15.446575 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-aa0f30e1-d622-40c1-908a-64ad759e5434 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:23:15.463262 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0f30e1-d622-40c1-908a-64ad759e5434 admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:15.484517 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 934/1885] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:23:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:23:15.497212 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383fc6dd-f228-4828-8a1e-a2af9653ad1a None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:15.498627 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-383fc6dd-f228-4828-8a1e-a2af9653ad1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:23:15.542998 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53149a7f-0a87-432f-a666-52d9c8ecaa70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:15.543613 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53149a7f-0a87-432f-a666-52d9c8ecaa70 admin admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:15.543901 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53149a7f-0a87-432f-a666-52d9c8ecaa70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:15.544254 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53149a7f-0a87-432f-a666-52d9c8ecaa70 admin admin] PATH_INFO: `/v3/roles` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:15.545182 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53149a7f-0a87-432f-a666-52d9c8ecaa70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:15.546781 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53149a7f-0a87-432f-a666-52d9c8ecaa70 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:15.557712 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 952/1886] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:23:15 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:23:15.560602 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383fc6dd-f228-4828-8a1e-a2af9653ad1a None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:15.562022 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53149a7f-0a87-432f-a666-52d9c8ecaa70 admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:15.564423 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f09eae-103f-4da3-b7f2-c96880db484c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0817c3320d5d41d8bba0550c8ae8dd60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:15.565071 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f09eae-103f-4da3-b7f2-c96880db484c service neutron] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:15.565870 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f09eae-103f-4da3-b7f2-c96880db484c service neutron] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:15.565870 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f09eae-103f-4da3-b7f2-c96880db484c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:15.571909 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53149a7f-0a87-432f-a666-52d9c8ecaa70 admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:15.576702 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f09eae-103f-4da3-b7f2-c96880db484c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:15.576848 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95be2a0-f6eb-4f87-80fb-856f2ed7fb6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:15.577548 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95be2a0-f6eb-4f87-80fb-856f2ed7fb6e admin admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:15.577841 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95be2a0-f6eb-4f87-80fb-856f2ed7fb6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:15.578123 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95be2a0-f6eb-4f87-80fb-856f2ed7fb6e admin admin] PATH_INFO: `/v3/roles` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:15.578218 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f09eae-103f-4da3-b7f2-c96880db484c service neutron] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:15.579343 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95be2a0-f6eb-4f87-80fb-856f2ed7fb6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:15.580857 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95be2a0-f6eb-4f87-80fb-856f2ed7fb6e admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:15.590373 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 935/1887] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:23:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 01:23:15.627643 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 953/1888] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:23:15 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 01:23:15.634105 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f09eae-103f-4da3-b7f2-c96880db484c service neutron] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:15.642584 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f09eae-103f-4da3-b7f2-c96880db484c service neutron] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:15.651719 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95bd312-ef72-4891-bd4f-52b2ddb658c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:15.652163 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95bd312-ef72-4891-bd4f-52b2ddb658c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:15.652450 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95bd312-ef72-4891-bd4f-52b2ddb658c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:15.652750 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95bd312-ef72-4891-bd4f-52b2ddb658c3 admin admin] PATH_INFO: `/v3/projects/2127f49c51164180ab4dfd834bab406f/users/a69eca1946824a148db97ae7f7a67eb8/roles/d0b9768f4d014766b9326387f2e11f34` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:15.677826 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95bd312-ef72-4891-bd4f-52b2ddb658c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2127f49c51164180ab4dfd834bab406f, user_id=a69eca1946824a148db97ae7f7a67eb8, role_id=d0b9768f4d014766b9326387f2e11f34)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:15.680156 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95bd312-ef72-4891-bd4f-52b2ddb658c3 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:15.794158 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 936/1889] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 01:23:15 2022] PUT /identity/v3/projects/2127f49c51164180ab4dfd834bab406f/users/a69eca1946824a148db97ae7f7a67eb8/roles/d0b9768f4d014766b9326387f2e11f34 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:23:15.804559 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce76657-d1a2-4b57-9964-c51fa64db648 None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:15.804939 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce76657-d1a2-4b57-9964-c51fa64db648 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:15.805338 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce76657-d1a2-4b57-9964-c51fa64db648 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:15.839112 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-2ce76657-d1a2-4b57-9964-c51fa64db648 None None] MFA Rules not processed for user `a69eca1946824a148db97ae7f7a67eb8`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:23:15.840568 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-2ce76657-d1a2-4b57-9964-c51fa64db648 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:23:15.932904 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 954/1890] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:23:15 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:23:15.944186 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95bd312-ef72-4891-bd4f-52b2ddb658c3 admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:15.946412 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-b95bd312-ef72-4891-bd4f-52b2ddb658c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:23:15.978687 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce76657-d1a2-4b57-9964-c51fa64db648 None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:15.980193 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-2ce76657-d1a2-4b57-9964-c51fa64db648 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:23:16.098471 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95bd312-ef72-4891-bd4f-52b2ddb658c3 admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:16.106416 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce76657-d1a2-4b57-9964-c51fa64db648 None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:16.109408 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aadb0fe-5734-4192-8ac3-03cb22384081 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8590c288028418ebe7a4163fd8039f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:16.110066 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aadb0fe-5734-4192-8ac3-03cb22384081 service glance] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:16.110353 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aadb0fe-5734-4192-8ac3-03cb22384081 service glance] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:16.112470 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aadb0fe-5734-4192-8ac3-03cb22384081 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:16.113420 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0534745f-cc93-4ae2-9818-e76555388292 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:16.114578 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0534745f-cc93-4ae2-9818-e76555388292 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:16.114898 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0534745f-cc93-4ae2-9818-e76555388292 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:16.115178 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0534745f-cc93-4ae2-9818-e76555388292 admin admin] PATH_INFO: `/v3/projects/523e95c967ac42809cb1b3799a238cb8` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:16.116389 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0534745f-cc93-4ae2-9818-e76555388292 admin admin] RBAC: Authorizing `identity:delete_project(project_id=523e95c967ac42809cb1b3799a238cb8)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:16.118949 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0534745f-cc93-4ae2-9818-e76555388292 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:16.151519 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aadb0fe-5734-4192-8ac3-03cb22384081 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:16.152945 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aadb0fe-5734-4192-8ac3-03cb22384081 service glance] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:16.166036 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 937/1891] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:23:15 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:23:16.166995 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aadb0fe-5734-4192-8ac3-03cb22384081 service glance] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:16.168479 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-4aadb0fe-5734-4192-8ac3-03cb22384081 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:23:16.183087 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aadb0fe-5734-4192-8ac3-03cb22384081 service glance] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:16.188281 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dc0296-45e5-4274-b235-11e80cf8b4c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:16.189083 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dc0296-45e5-4274-b235-11e80cf8b4c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:16.197000 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dc0296-45e5-4274-b235-11e80cf8b4c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:16.197000 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dc0296-45e5-4274-b235-11e80cf8b4c6 admin admin] PATH_INFO: `/v3/users/d0a8658f389d4367a1a52cfb09065835` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:16.197000 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41dc0296-45e5-4274-b235-11e80cf8b4c6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d0a8658f389d4367a1a52cfb09065835)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:16.197000 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41dc0296-45e5-4274-b235-11e80cf8b4c6 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:16.302566 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-0534745f-cc93-4ae2-9818-e76555388292 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '523e95c967ac42809cb1b3799a238cb8', 'request_id': 'req-0534745f-cc93-4ae2-9818-e76555388292'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:16.311226 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-0534745f-cc93-4ae2-9818-e76555388292 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '523e95c967ac42809cb1b3799a238cb8', 'request_id': 'req-0534745f-cc93-4ae2-9818-e76555388292'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:16.398642 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:23:16.400232 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-0534745f-cc93-4ae2-9818-e76555388292 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '523e95c967ac42809cb1b3799a238cb8', 'request_id': 'req-0534745f-cc93-4ae2-9818-e76555388292'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:16.416376 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-41dc0296-45e5-4274-b235-11e80cf8b4c6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd0a8658f389d4367a1a52cfb09065835'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:16.418797 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-0534745f-cc93-4ae2-9818-e76555388292 admin admin] The token cache is being invalidate because project 523e95c967ac42809cb1b3799a238cb8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81750) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 01:23:16.419153 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-0534745f-cc93-4ae2-9818-e76555388292 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:16.420007 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-0534745f-cc93-4ae2-9818-e76555388292 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:16.422400 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 955/1892] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 01:23:15 2022] DELETE /identity/v3/projects/523e95c967ac42809cb1b3799a238cb8 => generated 0 bytes in 445 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 01:23:16.423334 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0534745f-cc93-4ae2-9818-e76555388292 admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:16.424851 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-0534745f-cc93-4ae2-9818-e76555388292 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:23:16.481648 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-41dc0296-45e5-4274-b235-11e80cf8b4c6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd0a8658f389d4367a1a52cfb09065835'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:16.482423 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-41dc0296-45e5-4274-b235-11e80cf8b4c6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd0a8658f389d4367a1a52cfb09065835'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:16.491177 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-41dc0296-45e5-4274-b235-11e80cf8b4c6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd0a8658f389d4367a1a52cfb09065835'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:16.492130 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-41dc0296-45e5-4274-b235-11e80cf8b4c6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd0a8658f389d4367a1a52cfb09065835'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:16.515073 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0534745f-cc93-4ae2-9818-e76555388292 admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:16.526112 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 938/1893] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 01:23:16 2022] DELETE /identity/v3/users/d0a8658f389d4367a1a52cfb09065835 => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:23:16.527501 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36d75cf-2cce-41a5-ae15-082d8547f8fc None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:16.527640 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2128aaa-5527-43e9-b7a9-a0482bda2145 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:16.527804 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36d75cf-2cce-41a5-ae15-082d8547f8fc None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:16.528062 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36d75cf-2cce-41a5-ae15-082d8547f8fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:16.528447 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2128aaa-5527-43e9-b7a9-a0482bda2145 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:16.529149 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2128aaa-5527-43e9-b7a9-a0482bda2145 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:16.529510 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2128aaa-5527-43e9-b7a9-a0482bda2145 admin admin] PATH_INFO: `/v3/projects/f1bb404c17954bbfa4aa9d4d14ddea0f` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:16.531379 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2128aaa-5527-43e9-b7a9-a0482bda2145 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f1bb404c17954bbfa4aa9d4d14ddea0f)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:16.537308 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2128aaa-5527-43e9-b7a9-a0482bda2145 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:16.560102 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-e36d75cf-2cce-41a5-ae15-082d8547f8fc None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:23:16.562113 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-e36d75cf-2cce-41a5-ae15-082d8547f8fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:23:16.628945 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 939/1894] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:23:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:23:16.631304 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95455cd1-476b-4d90-a533-b1e7db9a0be0 None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:16.631600 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95455cd1-476b-4d90-a533-b1e7db9a0be0 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:16.631878 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95455cd1-476b-4d90-a533-b1e7db9a0be0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:16.656742 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-95455cd1-476b-4d90-a533-b1e7db9a0be0 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:23:16.658494 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-95455cd1-476b-4d90-a533-b1e7db9a0be0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:23:16.680036 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 940/1895] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:23:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:23:16.682262 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46a8666-1b49-488f-a0ef-ee77f80c3bd5 None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:16.683614 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46a8666-1b49-488f-a0ef-ee77f80c3bd5 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:16.683733 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46a8666-1b49-488f-a0ef-ee77f80c3bd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:16.704555 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-f2128aaa-5527-43e9-b7a9-a0482bda2145 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f1bb404c17954bbfa4aa9d4d14ddea0f', 'request_id': 'req-f2128aaa-5527-43e9-b7a9-a0482bda2145'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:16.710873 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-e46a8666-1b49-488f-a0ef-ee77f80c3bd5 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:23:16.712115 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-e46a8666-1b49-488f-a0ef-ee77f80c3bd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:23:16.712232 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-f2128aaa-5527-43e9-b7a9-a0482bda2145 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f1bb404c17954bbfa4aa9d4d14ddea0f', 'request_id': 'req-f2128aaa-5527-43e9-b7a9-a0482bda2145'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:16.787960 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 941/1896] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:23:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:23:16.789170 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46a8666-1b49-488f-a0ef-ee77f80c3bd5 None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:16.792252 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-e46a8666-1b49-488f-a0ef-ee77f80c3bd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:23:16.828103 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-f2128aaa-5527-43e9-b7a9-a0482bda2145 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f1bb404c17954bbfa4aa9d4d14ddea0f', 'request_id': 'req-f2128aaa-5527-43e9-b7a9-a0482bda2145'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:16.856124 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-f2128aaa-5527-43e9-b7a9-a0482bda2145 admin admin] The token cache is being invalidate because project f1bb404c17954bbfa4aa9d4d14ddea0f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81750) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 01:23:16.856298 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-f2128aaa-5527-43e9-b7a9-a0482bda2145 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:16.857113 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-f2128aaa-5527-43e9-b7a9-a0482bda2145 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:16.859225 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 956/1897] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 01:23:16 2022] DELETE /identity/v3/projects/f1bb404c17954bbfa4aa9d4d14ddea0f => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:23:16.860151 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2128aaa-5527-43e9-b7a9-a0482bda2145 admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:16.861622 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-f2128aaa-5527-43e9-b7a9-a0482bda2145 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:23:16.866828 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46a8666-1b49-488f-a0ef-ee77f80c3bd5 None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:16.871555 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ce3d91-b1a5-4d33-9ba6-02ec20b46c3e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0817c3320d5d41d8bba0550c8ae8dd60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:16.872182 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ce3d91-b1a5-4d33-9ba6-02ec20b46c3e service neutron] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:16.872511 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ce3d91-b1a5-4d33-9ba6-02ec20b46c3e service neutron] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:16.872825 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ce3d91-b1a5-4d33-9ba6-02ec20b46c3e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:16.875231 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-71ce3d91-b1a5-4d33-9ba6-02ec20b46c3e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:23:16.879038 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2128aaa-5527-43e9-b7a9-a0482bda2145 admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:16.885011 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b543819-7af6-46c9-8e81-2f5aed149d6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:16.885768 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b543819-7af6-46c9-8e81-2f5aed149d6c admin admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:16.886176 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b543819-7af6-46c9-8e81-2f5aed149d6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:16.886681 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b543819-7af6-46c9-8e81-2f5aed149d6c admin admin] PATH_INFO: `/v3/domains` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:16.887399 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b543819-7af6-46c9-8e81-2f5aed149d6c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:23:16.887713 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b543819-7af6-46c9-8e81-2f5aed149d6c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:16.889790 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b543819-7af6-46c9-8e81-2f5aed149d6c admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:16.889904 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ce3d91-b1a5-4d33-9ba6-02ec20b46c3e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:16.891285 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ce3d91-b1a5-4d33-9ba6-02ec20b46c3e service neutron] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:16.908463 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 942/1898] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:23:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:23:16.912591 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ce3d91-b1a5-4d33-9ba6-02ec20b46c3e service neutron] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:16.914118 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-71ce3d91-b1a5-4d33-9ba6-02ec20b46c3e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:23:16.929379 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ce3d91-b1a5-4d33-9ba6-02ec20b46c3e service neutron] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:16.931787 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 957/1899] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Aug 31 01:23:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:23:16.936050 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b543819-7af6-46c9-8e81-2f5aed149d6c admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:16.939878 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e53248-31bc-42d3-bd41-41021b384285 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:16.940531 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e53248-31bc-42d3-bd41-41021b384285 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:16.940867 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e53248-31bc-42d3-bd41-41021b384285 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:16.941180 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e53248-31bc-42d3-bd41-41021b384285 admin admin] PATH_INFO: `/v3/users/7998e2533226409cb62d5a33693757f1` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:16.942302 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e53248-31bc-42d3-bd41-41021b384285 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7998e2533226409cb62d5a33693757f1)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:16.944862 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e53248-31bc-42d3-bd41-41021b384285 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:16.955716 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b543819-7af6-46c9-8e81-2f5aed149d6c admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:16.962015 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36902a37-739f-4df4-9bc4-156e40edb8fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:16.964235 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36902a37-739f-4df4-9bc4-156e40edb8fa admin admin] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:16.964235 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36902a37-739f-4df4-9bc4-156e40edb8fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:16.964235 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36902a37-739f-4df4-9bc4-156e40edb8fa admin admin] PATH_INFO: `/v3/projects` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:16.964235 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36902a37-739f-4df4-9bc4-156e40edb8fa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:16.966352 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36902a37-739f-4df4-9bc4-156e40edb8fa admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:17.116987 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 958/1900] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 01:23:16 2022] POST /identity/v3/projects => generated 373 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 01:23:17.122542 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36902a37-739f-4df4-9bc4-156e40edb8fa admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:17.144290 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-90e53248-31bc-42d3-bd41-41021b384285 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7998e2533226409cb62d5a33693757f1'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:17.221514 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36902a37-739f-4df4-9bc4-156e40edb8fa admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:17.227310 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cd5e51-96d3-4e23-a60b-d8944ee3b821 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:17.228037 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cd5e51-96d3-4e23-a60b-d8944ee3b821 admin admin] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:17.228447 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cd5e51-96d3-4e23-a60b-d8944ee3b821 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:17.228768 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cd5e51-96d3-4e23-a60b-d8944ee3b821 admin admin] PATH_INFO: `/v3/users` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:17.229638 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cd5e51-96d3-4e23-a60b-d8944ee3b821 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:17.242656 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cd5e51-96d3-4e23-a60b-d8944ee3b821 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:17.272329 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-90e53248-31bc-42d3-bd41-41021b384285 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7998e2533226409cb62d5a33693757f1'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:17.272686 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-90e53248-31bc-42d3-bd41-41021b384285 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7998e2533226409cb62d5a33693757f1'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:17.284051 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-90e53248-31bc-42d3-bd41-41021b384285 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7998e2533226409cb62d5a33693757f1'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:17.284051 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-90e53248-31bc-42d3-bd41-41021b384285 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7998e2533226409cb62d5a33693757f1'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:17.304812 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 943/1901] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 01:23:16 2022] DELETE /identity/v3/users/7998e2533226409cb62d5a33693757f1 => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:23:17.347111 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 959/1902] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 01:23:17 2022] POST /identity/v3/users => generated 287 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 01:23:17.351150 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e53248-31bc-42d3-bd41-41021b384285 admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:17.352589 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-90e53248-31bc-42d3-bd41-41021b384285 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:23:17.419698 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e53248-31bc-42d3-bd41-41021b384285 admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:17.424753 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f5461e-c7a5-414d-877d-3e5ea7d8972b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:17.425372 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f5461e-c7a5-414d-877d-3e5ea7d8972b admin admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:17.425655 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f5461e-c7a5-414d-877d-3e5ea7d8972b admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:17.425929 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f5461e-c7a5-414d-877d-3e5ea7d8972b admin admin] PATH_INFO: `/v3/roles` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:17.426578 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f5461e-c7a5-414d-877d-3e5ea7d8972b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:17.429003 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f5461e-c7a5-414d-877d-3e5ea7d8972b admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:17.442248 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 944/1903] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:23:17 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:23:17.444599 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cd5e51-96d3-4e23-a60b-d8944ee3b821 admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:17.445285 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f5461e-c7a5-414d-877d-3e5ea7d8972b admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:17.447070 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-51f5461e-c7a5-414d-877d-3e5ea7d8972b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:23:17.455365 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cd5e51-96d3-4e23-a60b-d8944ee3b821 admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:17.462383 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f5461e-c7a5-414d-877d-3e5ea7d8972b admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:17.465560 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346dfb03-83b0-4f2b-934e-92019aa969b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:17.466667 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346dfb03-83b0-4f2b-934e-92019aa969b4 admin admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:17.467152 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346dfb03-83b0-4f2b-934e-92019aa969b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:17.467665 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346dfb03-83b0-4f2b-934e-92019aa969b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:17.468341 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0f6f6e-e5e1-4caa-a0c5-319a273bbfb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:17.469264 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0f6f6e-e5e1-4caa-a0c5-319a273bbfb6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:17.469701 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346dfb03-83b0-4f2b-934e-92019aa969b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:17.469814 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0f6f6e-e5e1-4caa-a0c5-319a273bbfb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:17.470202 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0f6f6e-e5e1-4caa-a0c5-319a273bbfb6 admin admin] PATH_INFO: `/v3/projects/690897ebec3e45c9abe97ba6f73b2b65` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:17.473159 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0f6f6e-e5e1-4caa-a0c5-319a273bbfb6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=690897ebec3e45c9abe97ba6f73b2b65)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:17.473289 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346dfb03-83b0-4f2b-934e-92019aa969b4 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:17.475807 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0f6f6e-e5e1-4caa-a0c5-319a273bbfb6 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:17.494222 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 960/1904] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:23:17 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:23:17.499634 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346dfb03-83b0-4f2b-934e-92019aa969b4 admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:17.511357 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346dfb03-83b0-4f2b-934e-92019aa969b4 admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:17.516120 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b48f0b-8ced-463f-9be8-8cc5118f38f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:17.516821 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b48f0b-8ced-463f-9be8-8cc5118f38f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:17.517112 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b48f0b-8ced-463f-9be8-8cc5118f38f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:17.517387 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b48f0b-8ced-463f-9be8-8cc5118f38f6 admin admin] PATH_INFO: `/v3/projects/1a4173ddac70420586849aa47642a9a3/users/91f71abcbc54454da9a52debf048ff0f/roles/d0b9768f4d014766b9326387f2e11f34` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:17.532826 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b48f0b-8ced-463f-9be8-8cc5118f38f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a4173ddac70420586849aa47642a9a3, user_id=91f71abcbc54454da9a52debf048ff0f, role_id=d0b9768f4d014766b9326387f2e11f34)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:17.535623 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b48f0b-8ced-463f-9be8-8cc5118f38f6 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:17.595706 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:23:17.642796 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 961/1905] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 01:23:17 2022] PUT /identity/v3/projects/1a4173ddac70420586849aa47642a9a3/users/91f71abcbc54454da9a52debf048ff0f/roles/d0b9768f4d014766b9326387f2e11f34 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 01:23:17.648384 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e47804-384e-47b4-a2a6-b4f98eb04e3f None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:17.648676 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e47804-384e-47b4-a2a6-b4f98eb04e3f None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:17.648982 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e47804-384e-47b4-a2a6-b4f98eb04e3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:17.655280 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-9a0f6f6e-e5e1-4caa-a0c5-319a273bbfb6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '690897ebec3e45c9abe97ba6f73b2b65', 'request_id': 'req-9a0f6f6e-e5e1-4caa-a0c5-319a273bbfb6'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:17.661055 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-9a0f6f6e-e5e1-4caa-a0c5-319a273bbfb6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '690897ebec3e45c9abe97ba6f73b2b65', 'request_id': 'req-9a0f6f6e-e5e1-4caa-a0c5-319a273bbfb6'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:17.668870 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-9a0f6f6e-e5e1-4caa-a0c5-319a273bbfb6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '690897ebec3e45c9abe97ba6f73b2b65', 'request_id': 'req-9a0f6f6e-e5e1-4caa-a0c5-319a273bbfb6'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:17.669349 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-06e47804-384e-47b4-a2a6-b4f98eb04e3f None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:23:17.670753 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-06e47804-384e-47b4-a2a6-b4f98eb04e3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:23:17.738622 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 962/1906] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:23:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 01:23:17.757051 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-9a0f6f6e-e5e1-4caa-a0c5-319a273bbfb6 admin admin] The token cache is being invalidate because project 690897ebec3e45c9abe97ba6f73b2b65 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81749) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 01:23:17.757486 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-9a0f6f6e-e5e1-4caa-a0c5-319a273bbfb6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:17.758319 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-9a0f6f6e-e5e1-4caa-a0c5-319a273bbfb6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:17.759922 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 945/1907] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 01:23:17 2022] DELETE /identity/v3/projects/690897ebec3e45c9abe97ba6f73b2b65 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 01:23:17.764283 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e47804-384e-47b4-a2a6-b4f98eb04e3f None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:17.775612 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e47804-384e-47b4-a2a6-b4f98eb04e3f None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:17.780989 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7575a5-10cb-4c94-9a2c-cdc542f492e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:17.781963 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7575a5-10cb-4c94-9a2c-cdc542f492e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:17.781963 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7575a5-10cb-4c94-9a2c-cdc542f492e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:17.782217 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7575a5-10cb-4c94-9a2c-cdc542f492e2 admin admin] PATH_INFO: `/v3/users/c04cfb38de834094a178a6815def316a` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:17.783320 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7575a5-10cb-4c94-9a2c-cdc542f492e2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c04cfb38de834094a178a6815def316a)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:17.786028 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0f6f6e-e5e1-4caa-a0c5-319a273bbfb6 admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:17.788076 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-9a0f6f6e-e5e1-4caa-a0c5-319a273bbfb6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:23:17.789792 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7575a5-10cb-4c94-9a2c-cdc542f492e2 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:17.899193 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0f6f6e-e5e1-4caa-a0c5-319a273bbfb6 admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:17.903058 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e079da-0864-42f2-9d07-24c6db88b56b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0817c3320d5d41d8bba0550c8ae8dd60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:17.903816 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e079da-0864-42f2-9d07-24c6db88b56b service neutron] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:17.904108 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e079da-0864-42f2-9d07-24c6db88b56b service neutron] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:17.904391 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e079da-0864-42f2-9d07-24c6db88b56b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:17.906686 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-49e079da-0864-42f2-9d07-24c6db88b56b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:23:17.921530 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e079da-0864-42f2-9d07-24c6db88b56b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:17.922981 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e079da-0864-42f2-9d07-24c6db88b56b service neutron] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:17.938178 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 946/1908] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:23:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:23:17.991430 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-ee7575a5-10cb-4c94-9a2c-cdc542f492e2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c04cfb38de834094a178a6815def316a'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:18.008805 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-ee7575a5-10cb-4c94-9a2c-cdc542f492e2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c04cfb38de834094a178a6815def316a'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:18.010412 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-ee7575a5-10cb-4c94-9a2c-cdc542f492e2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c04cfb38de834094a178a6815def316a'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:18.020896 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-ee7575a5-10cb-4c94-9a2c-cdc542f492e2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c04cfb38de834094a178a6815def316a'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:18.089505 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-ee7575a5-10cb-4c94-9a2c-cdc542f492e2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c04cfb38de834094a178a6815def316a'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:18.101550 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e079da-0864-42f2-9d07-24c6db88b56b service neutron] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:18.107158 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 963/1909] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 01:23:17 2022] DELETE /identity/v3/users/c04cfb38de834094a178a6815def316a => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:23:18.107310 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-49e079da-0864-42f2-9d07-24c6db88b56b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:23:18.133828 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e079da-0864-42f2-9d07-24c6db88b56b service neutron] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:18.241867 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2d374a-1a7e-4617-bf4b-848e3426bc0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:18.242864 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2d374a-1a7e-4617-bf4b-848e3426bc0a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:18.243031 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2d374a-1a7e-4617-bf4b-848e3426bc0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:18.243411 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2d374a-1a7e-4617-bf4b-848e3426bc0a admin admin] PATH_INFO: `/v3/projects/6d28bd8815444d80b7a82db2a8cf4be8` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:18.244709 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2d374a-1a7e-4617-bf4b-848e3426bc0a admin admin] RBAC: Authorizing `identity:delete_project(project_id=6d28bd8815444d80b7a82db2a8cf4be8)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:18.246946 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2d374a-1a7e-4617-bf4b-848e3426bc0a admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:18.401320 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-de2d374a-1a7e-4617-bf4b-848e3426bc0a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6d28bd8815444d80b7a82db2a8cf4be8', 'request_id': 'req-de2d374a-1a7e-4617-bf4b-848e3426bc0a'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:18.407901 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-de2d374a-1a7e-4617-bf4b-848e3426bc0a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6d28bd8815444d80b7a82db2a8cf4be8', 'request_id': 'req-de2d374a-1a7e-4617-bf4b-848e3426bc0a'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:18.416168 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-de2d374a-1a7e-4617-bf4b-848e3426bc0a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6d28bd8815444d80b7a82db2a8cf4be8', 'request_id': 'req-de2d374a-1a7e-4617-bf4b-848e3426bc0a'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:18.533909 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-de2d374a-1a7e-4617-bf4b-848e3426bc0a admin admin] The token cache is being invalidate because project 6d28bd8815444d80b7a82db2a8cf4be8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81749) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 01:23:18.535668 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-de2d374a-1a7e-4617-bf4b-848e3426bc0a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:18.536550 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-de2d374a-1a7e-4617-bf4b-848e3426bc0a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:18.543869 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3128df31-0165-4ec0-9d85-83b5b13e5034 None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:18.544390 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3128df31-0165-4ec0-9d85-83b5b13e5034 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:18.544845 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3128df31-0165-4ec0-9d85-83b5b13e5034 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:18.548136 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 947/1910] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 01:23:18 2022] DELETE /identity/v3/projects/6d28bd8815444d80b7a82db2a8cf4be8 => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:23:18.570200 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-3128df31-0165-4ec0-9d85-83b5b13e5034 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:23:18.571819 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-3128df31-0165-4ec0-9d85-83b5b13e5034 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:23:18.666579 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 964/1911] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:23:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:23:18.675787 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f70c0c8-e1ab-4f0e-b0d7-135cdb91ab56 None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:18.676178 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f70c0c8-e1ab-4f0e-b0d7-135cdb91ab56 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:18.676406 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f70c0c8-e1ab-4f0e-b0d7-135cdb91ab56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:18.701758 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-8f70c0c8-e1ab-4f0e-b0d7-135cdb91ab56 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:23:18.703236 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-8f70c0c8-e1ab-4f0e-b0d7-135cdb91ab56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:23:18.714836 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 948/1912] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:23:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:23:18.719325 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3128df31-0165-4ec0-9d85-83b5b13e5034 None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:18.733210 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3128df31-0165-4ec0-9d85-83b5b13e5034 None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:18.747158 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94ca583-479b-489e-a023-d1c156be3f4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:18.747912 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94ca583-479b-489e-a023-d1c156be3f4b admin admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:18.748137 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94ca583-479b-489e-a023-d1c156be3f4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:18.749099 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94ca583-479b-489e-a023-d1c156be3f4b admin admin] PATH_INFO: `/v3/domains` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:18.749351 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94ca583-479b-489e-a023-d1c156be3f4b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:23:18.749654 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94ca583-479b-489e-a023-d1c156be3f4b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:18.751406 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94ca583-479b-489e-a023-d1c156be3f4b admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:18.767077 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 965/1913] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Aug 31 01:23:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:23:18.771807 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f70c0c8-e1ab-4f0e-b0d7-135cdb91ab56 None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:18.781072 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f70c0c8-e1ab-4f0e-b0d7-135cdb91ab56 None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:18.789106 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7c07f8-21e9-42a8-8cc3-fe6080e62d1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:18.789728 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7c07f8-21e9-42a8-8cc3-fe6080e62d1e admin admin] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:18.790012 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7c07f8-21e9-42a8-8cc3-fe6080e62d1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:18.790292 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7c07f8-21e9-42a8-8cc3-fe6080e62d1e admin admin] PATH_INFO: `/v3/projects` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:18.790955 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7c07f8-21e9-42a8-8cc3-fe6080e62d1e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:18.793314 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7c07f8-21e9-42a8-8cc3-fe6080e62d1e admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:18.807703 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:23:18.823376 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94ca583-479b-489e-a023-d1c156be3f4b admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:18.828138 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-f94ca583-479b-489e-a023-d1c156be3f4b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:23:18.859140 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94ca583-479b-489e-a023-d1c156be3f4b admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:18.863910 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fc6399-79a3-4b29-a7e8-2fbdf676ab0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:18.864551 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fc6399-79a3-4b29-a7e8-2fbdf676ab0a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:18.865499 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fc6399-79a3-4b29-a7e8-2fbdf676ab0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:18.865773 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fc6399-79a3-4b29-a7e8-2fbdf676ab0a admin admin] PATH_INFO: `/v3/projects/fa5bbb1ad028460cbc060d0ec7065efd` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:18.867051 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fc6399-79a3-4b29-a7e8-2fbdf676ab0a admin admin] RBAC: Authorizing `identity:delete_project(project_id=fa5bbb1ad028460cbc060d0ec7065efd)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:18.869872 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fc6399-79a3-4b29-a7e8-2fbdf676ab0a admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:18.940556 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 949/1914] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 01:23:18 2022] POST /identity/v3/projects => generated 365 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:23:18.945202 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7c07f8-21e9-42a8-8cc3-fe6080e62d1e admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:19.015230 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7c07f8-21e9-42a8-8cc3-fe6080e62d1e admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:19.034899 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-e9fc6399-79a3-4b29-a7e8-2fbdf676ab0a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fa5bbb1ad028460cbc060d0ec7065efd', 'request_id': 'req-e9fc6399-79a3-4b29-a7e8-2fbdf676ab0a'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:19.045463 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-e9fc6399-79a3-4b29-a7e8-2fbdf676ab0a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fa5bbb1ad028460cbc060d0ec7065efd', 'request_id': 'req-e9fc6399-79a3-4b29-a7e8-2fbdf676ab0a'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:19.086728 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de071439-ed89-4c82-8a6a-13586fb3d3c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:19.087585 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de071439-ed89-4c82-8a6a-13586fb3d3c0 admin admin] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:19.087934 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de071439-ed89-4c82-8a6a-13586fb3d3c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:19.088288 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de071439-ed89-4c82-8a6a-13586fb3d3c0 admin admin] PATH_INFO: `/v3/users` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:19.089045 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de071439-ed89-4c82-8a6a-13586fb3d3c0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:19.094779 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de071439-ed89-4c82-8a6a-13586fb3d3c0 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:19.106563 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-e9fc6399-79a3-4b29-a7e8-2fbdf676ab0a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fa5bbb1ad028460cbc060d0ec7065efd', 'request_id': 'req-e9fc6399-79a3-4b29-a7e8-2fbdf676ab0a'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:19.121731 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-e9fc6399-79a3-4b29-a7e8-2fbdf676ab0a admin admin] The token cache is being invalidate because project fa5bbb1ad028460cbc060d0ec7065efd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81750) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 01:23:19.122030 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-e9fc6399-79a3-4b29-a7e8-2fbdf676ab0a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:19.122723 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-e9fc6399-79a3-4b29-a7e8-2fbdf676ab0a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:19.126458 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 966/1915] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 01:23:18 2022] DELETE /identity/v3/projects/fa5bbb1ad028460cbc060d0ec7065efd => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:23:19.131785 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02042410-d82b-4289-8b6f-13711b5f011f None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:19.132174 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02042410-d82b-4289-8b6f-13711b5f011f None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:19.132530 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02042410-d82b-4289-8b6f-13711b5f011f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:19.157402 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-02042410-d82b-4289-8b6f-13711b5f011f None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:23:19.158795 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-02042410-d82b-4289-8b6f-13711b5f011f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:23:19.174051 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 967/1916] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:23:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:23:19.178632 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 950/1917] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 01:23:18 2022] POST /identity/v3/users => generated 283 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 01:23:19.186334 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02042410-d82b-4289-8b6f-13711b5f011f None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:19.186472 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8232ea3d-7758-4cdd-a06e-edd424b986ff None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:19.186472 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8232ea3d-7758-4cdd-a06e-edd424b986ff None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:19.186786 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8232ea3d-7758-4cdd-a06e-edd424b986ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:19.187731 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-02042410-d82b-4289-8b6f-13711b5f011f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:23:19.200510 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02042410-d82b-4289-8b6f-13711b5f011f None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:19.209373 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0743267-f54e-487d-9849-901381316a67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:19.209373 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0743267-f54e-487d-9849-901381316a67 admin admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:19.209373 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0743267-f54e-487d-9849-901381316a67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:19.209651 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-8232ea3d-7758-4cdd-a06e-edd424b986ff None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:23:19.209737 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0743267-f54e-487d-9849-901381316a67 admin admin] PATH_INFO: `/v3/roles` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:19.209737 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0743267-f54e-487d-9849-901381316a67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:19.209914 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-8232ea3d-7758-4cdd-a06e-edd424b986ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:23:19.211651 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0743267-f54e-487d-9849-901381316a67 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:19.218386 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 951/1918] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:23:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:23:19.222467 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8232ea3d-7758-4cdd-a06e-edd424b986ff None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:19.226533 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 968/1919] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:23:19 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:23:19.233766 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8232ea3d-7758-4cdd-a06e-edd424b986ff None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:19.239120 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0743267-f54e-487d-9849-901381316a67 admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:19.243421 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3299a78-51af-4fd8-9450-a3a242e8f880 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:19.244047 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3299a78-51af-4fd8-9450-a3a242e8f880 admin admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:19.244332 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3299a78-51af-4fd8-9450-a3a242e8f880 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:19.245333 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3299a78-51af-4fd8-9450-a3a242e8f880 admin admin] PATH_INFO: `/v3/domains` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:19.246001 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3299a78-51af-4fd8-9450-a3a242e8f880 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:23:19.246310 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3299a78-51af-4fd8-9450-a3a242e8f880 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:19.248106 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3299a78-51af-4fd8-9450-a3a242e8f880 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:19.251999 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0743267-f54e-487d-9849-901381316a67 admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:19.257130 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edb3a54-018f-4f63-9eb9-4327aaf34d70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:19.257605 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edb3a54-018f-4f63-9eb9-4327aaf34d70 admin admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:19.257887 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edb3a54-018f-4f63-9eb9-4327aaf34d70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:19.258187 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edb3a54-018f-4f63-9eb9-4327aaf34d70 admin admin] PATH_INFO: `/v3/roles` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:19.258992 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edb3a54-018f-4f63-9eb9-4327aaf34d70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:19.260542 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 952/1920] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Aug 31 01:23:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 01:23:19.265126 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3299a78-51af-4fd8-9450-a3a242e8f880 admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:19.276440 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edb3a54-018f-4f63-9eb9-4327aaf34d70 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:19.277322 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3299a78-51af-4fd8-9450-a3a242e8f880 admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:19.282590 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295d77cd-754b-434d-a97a-d36c57aa90be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:19.283213 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295d77cd-754b-434d-a97a-d36c57aa90be admin admin] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:19.283501 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295d77cd-754b-434d-a97a-d36c57aa90be admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:19.283774 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295d77cd-754b-434d-a97a-d36c57aa90be admin admin] PATH_INFO: `/v3/projects` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:19.284785 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295d77cd-754b-434d-a97a-d36c57aa90be admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:19.286957 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295d77cd-754b-434d-a97a-d36c57aa90be admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:19.292626 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 969/1921] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:23:19 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:23:19.297597 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edb3a54-018f-4f63-9eb9-4327aaf34d70 admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:19.333070 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edb3a54-018f-4f63-9eb9-4327aaf34d70 admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:19.339824 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3072c2b7-2948-4715-a8be-54595d22d213 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:19.342799 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3072c2b7-2948-4715-a8be-54595d22d213 admin admin] REQUEST_METHOD: `PUT` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:19.343010 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3072c2b7-2948-4715-a8be-54595d22d213 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:19.343293 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3072c2b7-2948-4715-a8be-54595d22d213 admin admin] PATH_INFO: `/v3/projects/7e7373ed73c540eb80de520de253800a/users/016f7b40b0ce4c8297e65d943a65da4d/roles/d0b9768f4d014766b9326387f2e11f34` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:19.370851 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3072c2b7-2948-4715-a8be-54595d22d213 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e7373ed73c540eb80de520de253800a, user_id=016f7b40b0ce4c8297e65d943a65da4d, role_id=d0b9768f4d014766b9326387f2e11f34)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:19.374412 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3072c2b7-2948-4715-a8be-54595d22d213 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:19.392596 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 953/1922] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 01:23:19 2022] POST /identity/v3/projects => generated 385 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 01:23:19.396458 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295d77cd-754b-434d-a97a-d36c57aa90be admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:19.484561 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 970/1923] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 01:23:19 2022] PUT /identity/v3/projects/7e7373ed73c540eb80de520de253800a/users/016f7b40b0ce4c8297e65d943a65da4d/roles/d0b9768f4d014766b9326387f2e11f34 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:23:19.486036 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295d77cd-754b-434d-a97a-d36c57aa90be admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:19.503809 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0291aba6-a7a1-403e-be5a-a28862b7dfaf None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:19.503809 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0291aba6-a7a1-403e-be5a-a28862b7dfaf None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:19.503809 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0291aba6-a7a1-403e-be5a-a28862b7dfaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:19.567447 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-0291aba6-a7a1-403e-be5a-a28862b7dfaf None None] MFA Rules not processed for user `016f7b40b0ce4c8297e65d943a65da4d`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:23:19.570001 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-0291aba6-a7a1-403e-be5a-a28862b7dfaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:23:19.577507 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:23:19.618683 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e204cf4-967c-4c3f-b49b-da953d78e373 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:19.619593 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e204cf4-967c-4c3f-b49b-da953d78e373 admin admin] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:19.620094 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e204cf4-967c-4c3f-b49b-da953d78e373 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:19.620476 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e204cf4-967c-4c3f-b49b-da953d78e373 admin admin] PATH_INFO: `/v3/users` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:19.621235 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e204cf4-967c-4c3f-b49b-da953d78e373 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:19.625297 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e204cf4-967c-4c3f-b49b-da953d78e373 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:19.670715 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 971/1924] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:23:19 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:23:19.675823 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0291aba6-a7a1-403e-be5a-a28862b7dfaf None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:19.692168 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0291aba6-a7a1-403e-be5a-a28862b7dfaf None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:19.698225 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbd104c-5834-468a-959c-9b187b64c500 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:19.699717 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbd104c-5834-468a-959c-9b187b64c500 admin admin] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:19.700199 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbd104c-5834-468a-959c-9b187b64c500 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:19.700654 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbd104c-5834-468a-959c-9b187b64c500 admin admin] PATH_INFO: `/v3/projects` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:19.701548 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbd104c-5834-468a-959c-9b187b64c500 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:19.703697 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbd104c-5834-468a-959c-9b187b64c500 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:19.762503 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 954/1925] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 01:23:19 2022] POST /identity/v3/users => generated 293 bytes in 366 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:23:19.763710 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e204cf4-967c-4c3f-b49b-da953d78e373 admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:19.783010 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e204cf4-967c-4c3f-b49b-da953d78e373 admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:19.788121 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb19251a-2b03-4d61-8d58-f4a56f178b46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:19.788687 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb19251a-2b03-4d61-8d58-f4a56f178b46 admin admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:19.788996 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb19251a-2b03-4d61-8d58-f4a56f178b46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:19.789299 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb19251a-2b03-4d61-8d58-f4a56f178b46 admin admin] PATH_INFO: `/v3/roles` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:19.789948 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb19251a-2b03-4d61-8d58-f4a56f178b46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:19.792168 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb19251a-2b03-4d61-8d58-f4a56f178b46 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:19.808592 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 955/1926] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:23:19 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:23:19.809711 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb19251a-2b03-4d61-8d58-f4a56f178b46 admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:19.826317 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb19251a-2b03-4d61-8d58-f4a56f178b46 admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:19.828171 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 972/1927] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 01:23:19 2022] POST /identity/v3/projects => generated 363 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 01:23:19.832403 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbd104c-5834-468a-959c-9b187b64c500 admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:19.916393 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbd104c-5834-468a-959c-9b187b64c500 admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:19.923222 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7825161-c911-466b-b5fd-b84a859b035d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:19.925467 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7825161-c911-466b-b5fd-b84a859b035d admin admin] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:19.925972 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7825161-c911-466b-b5fd-b84a859b035d admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:19.926627 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7825161-c911-466b-b5fd-b84a859b035d admin admin] PATH_INFO: `/v3/users` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:19.928388 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7825161-c911-466b-b5fd-b84a859b035d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:19.930574 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7825161-c911-466b-b5fd-b84a859b035d admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:19.933056 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fcf125-c699-4a3d-a93b-bc033290fb41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:19.934021 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fcf125-c699-4a3d-a93b-bc033290fb41 admin admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:19.934509 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fcf125-c699-4a3d-a93b-bc033290fb41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:19.935609 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fcf125-c699-4a3d-a93b-bc033290fb41 admin admin] PATH_INFO: `/v3/roles` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:19.936420 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fcf125-c699-4a3d-a93b-bc033290fb41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:19.941077 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fcf125-c699-4a3d-a93b-bc033290fb41 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:19.954378 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 956/1928] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:23:19 2022] GET /identity/v3/roles => generated 1420 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:23:19.955901 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb54170e-83ca-474c-a48b-d60f9df1f0ec None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:19.956325 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb54170e-83ca-474c-a48b-d60f9df1f0ec None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:19.956770 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb54170e-83ca-474c-a48b-d60f9df1f0ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:19.988290 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-eb54170e-83ca-474c-a48b-d60f9df1f0ec None None] MFA Rules not processed for user `91f71abcbc54454da9a52debf048ff0f`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:23:19.990140 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-eb54170e-83ca-474c-a48b-d60f9df1f0ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:23:20.081141 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 973/1929] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 01:23:19 2022] POST /identity/v3/users => generated 288 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:23:20.082389 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7825161-c911-466b-b5fd-b84a859b035d admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:20.092328 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 957/1930] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:23:19 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:23:20.093456 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb54170e-83ca-474c-a48b-d60f9df1f0ec None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:20.094759 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7825161-c911-466b-b5fd-b84a859b035d admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:20.105325 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb54170e-83ca-474c-a48b-d60f9df1f0ec None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:20.105511 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc608c8-978c-46cb-8855-cc93701b6df1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:20.107962 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc608c8-978c-46cb-8855-cc93701b6df1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:20.108182 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc608c8-978c-46cb-8855-cc93701b6df1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:20.112472 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc608c8-978c-46cb-8855-cc93701b6df1 admin admin] PATH_INFO: `/v3/projects/c0719f0fb98944edbdb5efa1691de44c/users/ebbed63bf7f34d3e99407f42d5b75832/roles/d0b9768f4d014766b9326387f2e11f34` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:20.112612 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67613d95-849c-401a-9e25-4f69548ece73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:20.112612 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67613d95-849c-401a-9e25-4f69548ece73 admin admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:20.112612 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67613d95-849c-401a-9e25-4f69548ece73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:20.112612 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67613d95-849c-401a-9e25-4f69548ece73 admin admin] PATH_INFO: `/v3/roles` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:20.113439 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67613d95-849c-401a-9e25-4f69548ece73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:20.115760 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67613d95-849c-401a-9e25-4f69548ece73 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:20.128321 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 958/1931] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:23:20 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:23:20.129330 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67613d95-849c-401a-9e25-4f69548ece73 admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:20.140097 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-67613d95-849c-401a-9e25-4f69548ece73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:23:20.142729 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc608c8-978c-46cb-8855-cc93701b6df1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c0719f0fb98944edbdb5efa1691de44c, user_id=ebbed63bf7f34d3e99407f42d5b75832, role_id=d0b9768f4d014766b9326387f2e11f34)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:20.146078 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc608c8-978c-46cb-8855-cc93701b6df1 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:20.155426 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67613d95-849c-401a-9e25-4f69548ece73 admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:20.162590 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6841df91-ce0e-45c5-ace9-28088374a242 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:20.163278 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6841df91-ce0e-45c5-ace9-28088374a242 admin admin] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:20.163569 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6841df91-ce0e-45c5-ace9-28088374a242 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:20.163843 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6841df91-ce0e-45c5-ace9-28088374a242 admin admin] PATH_INFO: `/v3/projects` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:20.167122 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6841df91-ce0e-45c5-ace9-28088374a242 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:20.172618 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6841df91-ce0e-45c5-ace9-28088374a242 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:20.226902 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 974/1932] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 01:23:20 2022] PUT /identity/v3/projects/c0719f0fb98944edbdb5efa1691de44c/users/ebbed63bf7f34d3e99407f42d5b75832/roles/d0b9768f4d014766b9326387f2e11f34 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:23:20.236938 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc608c8-978c-46cb-8855-cc93701b6df1 admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:20.264540 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 959/1933] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 01:23:20 2022] POST /identity/v3/projects => generated 371 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:23:20.264540 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e49d39-9300-46a6-864e-722d095e4554 None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:20.264540 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e49d39-9300-46a6-864e-722d095e4554 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:20.264540 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e49d39-9300-46a6-864e-722d095e4554 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:20.290824 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-c2e49d39-9300-46a6-864e-722d095e4554 None None] MFA Rules not processed for user `ebbed63bf7f34d3e99407f42d5b75832`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:23:20.292026 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-c2e49d39-9300-46a6-864e-722d095e4554 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:23:20.296265 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc608c8-978c-46cb-8855-cc93701b6df1 admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:20.353941 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846eb3af-8951-49f7-b2a9-c1367d4e8a90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:20.354678 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846eb3af-8951-49f7-b2a9-c1367d4e8a90 admin admin] REQUEST_METHOD: `PUT` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:20.355028 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846eb3af-8951-49f7-b2a9-c1367d4e8a90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:20.355474 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846eb3af-8951-49f7-b2a9-c1367d4e8a90 admin admin] PATH_INFO: `/v3/projects/321e86b471354116b0847bb7c8625664/users/1709a1e94c0e4979977cd6d30d418a69/roles/d009411496b8452b88e77ffb8db015a7` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:20.371427 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 960/1934] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:23:20 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:23:20.372101 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e49d39-9300-46a6-864e-722d095e4554 None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:20.387167 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846eb3af-8951-49f7-b2a9-c1367d4e8a90 admin admin] RBAC: Authorizing `identity:create_grant(project_id=321e86b471354116b0847bb7c8625664, user_id=1709a1e94c0e4979977cd6d30d418a69, role_id=d009411496b8452b88e77ffb8db015a7)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:20.390000 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846eb3af-8951-49f7-b2a9-c1367d4e8a90 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:20.392418 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e49d39-9300-46a6-864e-722d095e4554 None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:20.401491 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f01ec1c-77f7-4c5e-86bf-32ca25679062 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:20.409910 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f01ec1c-77f7-4c5e-86bf-32ca25679062 admin admin] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:20.409910 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f01ec1c-77f7-4c5e-86bf-32ca25679062 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:20.409910 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f01ec1c-77f7-4c5e-86bf-32ca25679062 admin admin] PATH_INFO: `/v3/users` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:20.409910 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f01ec1c-77f7-4c5e-86bf-32ca25679062 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:20.409910 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f01ec1c-77f7-4c5e-86bf-32ca25679062 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:20.449678 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 975/1935] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 01:23:20 2022] PUT /identity/v3/projects/321e86b471354116b0847bb7c8625664/users/1709a1e94c0e4979977cd6d30d418a69/roles/d009411496b8452b88e77ffb8db015a7 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:23:20.450870 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846eb3af-8951-49f7-b2a9-c1367d4e8a90 admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:20.521664 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846eb3af-8951-49f7-b2a9-c1367d4e8a90 admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:20.527056 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 961/1936] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 01:23:20 2022] POST /identity/v3/users => generated 292 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:23:20.527056 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f01ec1c-77f7-4c5e-86bf-32ca25679062 admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:20.529270 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2da8f46-953b-43c7-a964-8a733b1325a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:20.529931 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2da8f46-953b-43c7-a964-8a733b1325a4 admin admin] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:20.530207 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2da8f46-953b-43c7-a964-8a733b1325a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:20.530481 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2da8f46-953b-43c7-a964-8a733b1325a4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:20.531707 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2da8f46-953b-43c7-a964-8a733b1325a4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:20.533281 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2da8f46-953b-43c7-a964-8a733b1325a4 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:20.535407 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f01ec1c-77f7-4c5e-86bf-32ca25679062 admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:20.541809 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25bfe6f-3872-4a92-b304-e9bdb47159d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:20.542484 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25bfe6f-3872-4a92-b304-e9bdb47159d9 admin admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:20.542775 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25bfe6f-3872-4a92-b304-e9bdb47159d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:20.545596 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25bfe6f-3872-4a92-b304-e9bdb47159d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:20.545596 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25bfe6f-3872-4a92-b304-e9bdb47159d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:20.545596 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25bfe6f-3872-4a92-b304-e9bdb47159d9 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:20.571695 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 962/1937] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:23:20 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:23:20.572879 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25bfe6f-3872-4a92-b304-e9bdb47159d9 admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:20.605081 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25bfe6f-3872-4a92-b304-e9bdb47159d9 admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:20.611661 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c6b674-6c20-4f47-aeb7-2230f9bda0c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:20.612944 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c6b674-6c20-4f47-aeb7-2230f9bda0c2 admin admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:20.613473 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c6b674-6c20-4f47-aeb7-2230f9bda0c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:20.613959 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c6b674-6c20-4f47-aeb7-2230f9bda0c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:20.614997 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c6b674-6c20-4f47-aeb7-2230f9bda0c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:20.616870 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 976/1938] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 01:23:20 2022] POST /identity/v3/projects => generated 383 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:23:20.617236 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c6b674-6c20-4f47-aeb7-2230f9bda0c2 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:20.617694 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2da8f46-953b-43c7-a964-8a733b1325a4 admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:20.626571 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:23:20.632021 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 963/1939] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:23:20 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:23:20.633274 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c6b674-6c20-4f47-aeb7-2230f9bda0c2 admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:20.647883 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c6b674-6c20-4f47-aeb7-2230f9bda0c2 admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:20.737968 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fef0bb8-5e1b-4c33-b0aa-b5c4e00eff3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:20.739089 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fef0bb8-5e1b-4c33-b0aa-b5c4e00eff3d admin admin] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:20.739621 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fef0bb8-5e1b-4c33-b0aa-b5c4e00eff3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:20.740743 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fef0bb8-5e1b-4c33-b0aa-b5c4e00eff3d admin admin] PATH_INFO: `/v3/users` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:20.742118 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fef0bb8-5e1b-4c33-b0aa-b5c4e00eff3d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:20.747053 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fef0bb8-5e1b-4c33-b0aa-b5c4e00eff3d admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:20.787499 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2da8f46-953b-43c7-a964-8a733b1325a4 admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:20.793506 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79365ee9-6a75-4953-bed9-b02a0380b1cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:20.794369 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79365ee9-6a75-4953-bed9-b02a0380b1cf admin admin] REQUEST_METHOD: `PUT` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:20.794760 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79365ee9-6a75-4953-bed9-b02a0380b1cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:20.795139 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79365ee9-6a75-4953-bed9-b02a0380b1cf admin admin] PATH_INFO: `/v3/projects/321e86b471354116b0847bb7c8625664/users/1709a1e94c0e4979977cd6d30d418a69/roles/d009411496b8452b88e77ffb8db015a7` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:20.802584 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79365ee9-6a75-4953-bed9-b02a0380b1cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=321e86b471354116b0847bb7c8625664, user_id=1709a1e94c0e4979977cd6d30d418a69, role_id=d009411496b8452b88e77ffb8db015a7)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:20.805563 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79365ee9-6a75-4953-bed9-b02a0380b1cf admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:20.823443 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 977/1940] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 01:23:20 2022] PUT /identity/v3/projects/321e86b471354116b0847bb7c8625664/users/1709a1e94c0e4979977cd6d30d418a69/roles/d009411496b8452b88e77ffb8db015a7 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:23:20.824527 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79365ee9-6a75-4953-bed9-b02a0380b1cf admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:20.842001 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 964/1941] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 01:23:20 2022] POST /identity/v3/users => generated 298 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:23:20.843342 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a19787-ea33-483f-b2bc-c1a2411a868e None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:20.843635 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a19787-ea33-483f-b2bc-c1a2411a868e None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:20.843916 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a19787-ea33-483f-b2bc-c1a2411a868e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:20.893938 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-54a19787-ea33-483f-b2bc-c1a2411a868e None None] MFA Rules not processed for user `1709a1e94c0e4979977cd6d30d418a69`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:23:20.895201 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-54a19787-ea33-483f-b2bc-c1a2411a868e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:23:20.945194 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79365ee9-6a75-4953-bed9-b02a0380b1cf admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:20.950716 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3e3716-8136-4646-a8f4-3066e5957caa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:20.951415 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3e3716-8136-4646-a8f4-3066e5957caa admin admin] REQUEST_METHOD: `PUT` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:20.951735 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3e3716-8136-4646-a8f4-3066e5957caa admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:20.952101 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3e3716-8136-4646-a8f4-3066e5957caa admin admin] PATH_INFO: `/v3/projects/8fa93a330f004af585babdddf27ab6dc/users/fbdd2d19767648d78f4bd33dbc03ddd8/roles/d009411496b8452b88e77ffb8db015a7` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:20.971850 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3e3716-8136-4646-a8f4-3066e5957caa admin admin] RBAC: Authorizing `identity:create_grant(project_id=8fa93a330f004af585babdddf27ab6dc, user_id=fbdd2d19767648d78f4bd33dbc03ddd8, role_id=d009411496b8452b88e77ffb8db015a7)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:20.978201 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3e3716-8136-4646-a8f4-3066e5957caa admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:21.027145 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 965/1942] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:23:20 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:23:21.028124 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a19787-ea33-483f-b2bc-c1a2411a868e None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:21.039876 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a19787-ea33-483f-b2bc-c1a2411a868e None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:21.056205 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 978/1943] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 01:23:20 2022] PUT /identity/v3/projects/8fa93a330f004af585babdddf27ab6dc/users/fbdd2d19767648d78f4bd33dbc03ddd8/roles/d009411496b8452b88e77ffb8db015a7 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:23:21.057700 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3e3716-8136-4646-a8f4-3066e5957caa admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:21.063465 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-6f3e3716-8136-4646-a8f4-3066e5957caa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:23:21.137537 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002335c0-77d9-413b-be7b-be804aead39b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:21.138129 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002335c0-77d9-413b-be7b-be804aead39b admin admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:21.138450 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002335c0-77d9-413b-be7b-be804aead39b admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:21.140066 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002335c0-77d9-413b-be7b-be804aead39b admin admin] PATH_INFO: `/v3/roles` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:21.140813 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002335c0-77d9-413b-be7b-be804aead39b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:21.144535 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002335c0-77d9-413b-be7b-be804aead39b admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:21.153944 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3e3716-8136-4646-a8f4-3066e5957caa admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:21.159100 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebbce58-09c1-4b66-8ae8-a63e442d3488 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5c8fc69c518412ba1aad61a9f4f558c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:21.160279 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebbce58-09c1-4b66-8ae8-a63e442d3488 service cinder] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:21.161082 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebbce58-09c1-4b66-8ae8-a63e442d3488 service cinder] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:21.161745 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebbce58-09c1-4b66-8ae8-a63e442d3488 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:21.169081 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 966/1944] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:23:21 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:23:21.169081 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002335c0-77d9-413b-be7b-be804aead39b admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:21.188216 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002335c0-77d9-413b-be7b-be804aead39b admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:21.202553 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737b5fc6-e434-4406-a0f6-1685c08d233c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:21.204356 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737b5fc6-e434-4406-a0f6-1685c08d233c admin admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:21.204356 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737b5fc6-e434-4406-a0f6-1685c08d233c admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:21.204804 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737b5fc6-e434-4406-a0f6-1685c08d233c admin admin] PATH_INFO: `/v3/roles` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:21.206639 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737b5fc6-e434-4406-a0f6-1685c08d233c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:21.208934 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737b5fc6-e434-4406-a0f6-1685c08d233c admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:21.223216 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 967/1945] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:23:21 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:23:21.225098 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737b5fc6-e434-4406-a0f6-1685c08d233c admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:21.240014 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737b5fc6-e434-4406-a0f6-1685c08d233c admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:21.251055 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e558256d-f2fc-47b7-a944-3e9f76c0ac15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:21.252305 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e558256d-f2fc-47b7-a944-3e9f76c0ac15 admin admin] REQUEST_METHOD: `PUT` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:21.252652 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e558256d-f2fc-47b7-a944-3e9f76c0ac15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:21.253222 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e558256d-f2fc-47b7-a944-3e9f76c0ac15 admin admin] PATH_INFO: `/v3/projects/45529684e17a43539f53e2a57142428c/users/132dca16d40c467682619cfad463d274/roles/d009411496b8452b88e77ffb8db015a7` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:21.274621 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e558256d-f2fc-47b7-a944-3e9f76c0ac15 admin admin] RBAC: Authorizing `identity:create_grant(project_id=45529684e17a43539f53e2a57142428c, user_id=132dca16d40c467682619cfad463d274, role_id=d009411496b8452b88e77ffb8db015a7)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:21.278112 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e558256d-f2fc-47b7-a944-3e9f76c0ac15 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:21.288407 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebbce58-09c1-4b66-8ae8-a63e442d3488 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:21.293074 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebbce58-09c1-4b66-8ae8-a63e442d3488 service cinder] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:21.306922 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:23:21.316944 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 979/1946] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:23:21 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 260 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:23:21.318562 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebbce58-09c1-4b66-8ae8-a63e442d3488 service cinder] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:21.341675 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebbce58-09c1-4b66-8ae8-a63e442d3488 service cinder] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:21.353280 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e9e778-6b78-4b4a-be4b-394cf7f161c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:21.357827 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e9e778-6b78-4b4a-be4b-394cf7f161c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:21.358119 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e9e778-6b78-4b4a-be4b-394cf7f161c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:21.358403 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e9e778-6b78-4b4a-be4b-394cf7f161c9 admin admin] PATH_INFO: `/v3/projects/8fa93a330f004af585babdddf27ab6dc/users/fbdd2d19767648d78f4bd33dbc03ddd8/roles/d009411496b8452b88e77ffb8db015a7` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:21.360691 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e9e778-6b78-4b4a-be4b-394cf7f161c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8fa93a330f004af585babdddf27ab6dc, user_id=fbdd2d19767648d78f4bd33dbc03ddd8, role_id=d009411496b8452b88e77ffb8db015a7)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:21.368020 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e9e778-6b78-4b4a-be4b-394cf7f161c9 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:21.368614 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 968/1947] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 01:23:21 2022] PUT /identity/v3/projects/45529684e17a43539f53e2a57142428c/users/132dca16d40c467682619cfad463d274/roles/d009411496b8452b88e77ffb8db015a7 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:23:21.369450 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e558256d-f2fc-47b7-a944-3e9f76c0ac15 admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:21.380970 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 980/1948] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 01:23:21 2022] PUT /identity/v3/projects/8fa93a330f004af585babdddf27ab6dc/users/fbdd2d19767648d78f4bd33dbc03ddd8/roles/d009411496b8452b88e77ffb8db015a7 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:23:21.505975 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e558256d-f2fc-47b7-a944-3e9f76c0ac15 admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:21.581569 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4641068c-a54c-4fcc-81f9-bf5d0a536776 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:21.582202 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4641068c-a54c-4fcc-81f9-bf5d0a536776 admin admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:21.582621 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4641068c-a54c-4fcc-81f9-bf5d0a536776 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:21.583805 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4641068c-a54c-4fcc-81f9-bf5d0a536776 admin admin] PATH_INFO: `/v3/roles` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:21.584237 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4641068c-a54c-4fcc-81f9-bf5d0a536776 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:21.602927 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4641068c-a54c-4fcc-81f9-bf5d0a536776 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:21.612020 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e9e778-6b78-4b4a-be4b-394cf7f161c9 admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:21.614196 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-98e9e778-6b78-4b4a-be4b-394cf7f161c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:23:21.625559 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 969/1949] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:23:21 2022] GET /identity/v3/roles => generated 1420 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 01:23:21.626550 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4641068c-a54c-4fcc-81f9-bf5d0a536776 admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:21.644104 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4641068c-a54c-4fcc-81f9-bf5d0a536776 admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:21.660548 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5a36ba-6668-41ac-af02-af89c7b1190b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:21.661961 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5a36ba-6668-41ac-af02-af89c7b1190b admin admin] REQUEST_METHOD: `PUT` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:21.662249 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5a36ba-6668-41ac-af02-af89c7b1190b admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:21.662531 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5a36ba-6668-41ac-af02-af89c7b1190b admin admin] PATH_INFO: `/v3/projects/45529684e17a43539f53e2a57142428c/users/132dca16d40c467682619cfad463d274/roles/d009411496b8452b88e77ffb8db015a7` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:21.667340 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5a36ba-6668-41ac-af02-af89c7b1190b admin admin] RBAC: Authorizing `identity:create_grant(project_id=45529684e17a43539f53e2a57142428c, user_id=132dca16d40c467682619cfad463d274, role_id=d009411496b8452b88e77ffb8db015a7)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:21.679756 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5a36ba-6668-41ac-af02-af89c7b1190b admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:21.696448 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e9e778-6b78-4b4a-be4b-394cf7f161c9 admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:21.700662 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf8edc2-09be-4d2a-9bc2-bdc6422256af service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5c8fc69c518412ba1aad61a9f4f558c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:21.701319 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf8edc2-09be-4d2a-9bc2-bdc6422256af service cinder] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:21.701600 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf8edc2-09be-4d2a-9bc2-bdc6422256af service cinder] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:21.702230 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf8edc2-09be-4d2a-9bc2-bdc6422256af service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:21.706285 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:23:21.730539 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 970/1950] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 01:23:21 2022] PUT /identity/v3/projects/45529684e17a43539f53e2a57142428c/users/132dca16d40c467682619cfad463d274/roles/d009411496b8452b88e77ffb8db015a7 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:23:21.732293 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4aa192-db2d-48bb-a0d4-92a6ad80b69c None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:21.732600 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4aa192-db2d-48bb-a0d4-92a6ad80b69c None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:21.732963 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4aa192-db2d-48bb-a0d4-92a6ad80b69c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:21.794886 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-3a4aa192-db2d-48bb-a0d4-92a6ad80b69c None None] MFA Rules not processed for user `132dca16d40c467682619cfad463d274`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:23:21.794886 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-3a4aa192-db2d-48bb-a0d4-92a6ad80b69c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:23:21.814618 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf8edc2-09be-4d2a-9bc2-bdc6422256af service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:21.816591 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf8edc2-09be-4d2a-9bc2-bdc6422256af service cinder] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:21.838213 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 981/1951] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:23:21 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:23:21.945837 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 971/1952] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:23:21 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:23:21.982673 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf8edc2-09be-4d2a-9bc2-bdc6422256af service cinder] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:21.994311 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf8edc2-09be-4d2a-9bc2-bdc6422256af service cinder] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:22.090058 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2794102a-bc40-4d2e-811e-371d02204fcd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5c8fc69c518412ba1aad61a9f4f558c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:22.091289 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2794102a-bc40-4d2e-811e-371d02204fcd service cinder] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:22.091814 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2794102a-bc40-4d2e-811e-371d02204fcd service cinder] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:22.092337 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2794102a-bc40-4d2e-811e-371d02204fcd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:22.107225 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2794102a-bc40-4d2e-811e-371d02204fcd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:22.108940 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2794102a-bc40-4d2e-811e-371d02204fcd service cinder] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:22.124864 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 982/1953] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:23:21 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:23:22.203653 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:23:23.493853 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4aa192-db2d-48bb-a0d4-92a6ad80b69c None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:23.510082 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:23:23.600820 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4aa192-db2d-48bb-a0d4-92a6ad80b69c None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:23.609062 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71890128-25f0-4aa0-b8b0-dcd603865285 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:23.609857 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71890128-25f0-4aa0-b8b0-dcd603865285 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:23.610264 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71890128-25f0-4aa0-b8b0-dcd603865285 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:23.610620 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71890128-25f0-4aa0-b8b0-dcd603865285 admin admin] PATH_INFO: `/v3/users/ebbed63bf7f34d3e99407f42d5b75832` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:23.612142 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71890128-25f0-4aa0-b8b0-dcd603865285 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ebbed63bf7f34d3e99407f42d5b75832)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:23.614388 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71890128-25f0-4aa0-b8b0-dcd603865285 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:23.628157 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e17673-285d-465b-983c-b67e0ad362b0 None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:23.628359 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e17673-285d-465b-983c-b67e0ad362b0 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:23.628751 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e17673-285d-465b-983c-b67e0ad362b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:23.666730 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-c7e17673-285d-465b-983c-b67e0ad362b0 None None] MFA Rules not processed for user `fbdd2d19767648d78f4bd33dbc03ddd8`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:23:23.669158 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-c7e17673-285d-465b-983c-b67e0ad362b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:23:23.755049 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 983/1954] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:23:23 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:23:23.769397 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e17673-285d-465b-983c-b67e0ad362b0 None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:23.782141 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e17673-285d-465b-983c-b67e0ad362b0 None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:23.789291 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e7c290-ec90-447b-b5c4-50b699714fd5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5c8fc69c518412ba1aad61a9f4f558c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:23.790189 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e7c290-ec90-447b-b5c4-50b699714fd5 service cinder] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:23.790551 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e7c290-ec90-447b-b5c4-50b699714fd5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:23.790943 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e7c290-ec90-447b-b5c4-50b699714fd5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:23.859891 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e7c290-ec90-447b-b5c4-50b699714fd5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:23.861486 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e7c290-ec90-447b-b5c4-50b699714fd5 service cinder] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:23.868653 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-71890128-25f0-4aa0-b8b0-dcd603865285 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ebbed63bf7f34d3e99407f42d5b75832'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:23.871870 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 984/1955] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:23:23 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:23:23.980079 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-71890128-25f0-4aa0-b8b0-dcd603865285 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ebbed63bf7f34d3e99407f42d5b75832'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:23.980928 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-71890128-25f0-4aa0-b8b0-dcd603865285 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ebbed63bf7f34d3e99407f42d5b75832'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:23.990035 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-71890128-25f0-4aa0-b8b0-dcd603865285 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ebbed63bf7f34d3e99407f42d5b75832'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:23.991009 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-71890128-25f0-4aa0-b8b0-dcd603865285 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ebbed63bf7f34d3e99407f42d5b75832'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:24.012779 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 972/1956] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 01:23:23 2022] DELETE /identity/v3/users/ebbed63bf7f34d3e99407f42d5b75832 => generated 0 bytes in 520 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:23:24.017498 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80aae89b-1343-4a99-983c-0b79d93d9331 None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:24.017817 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80aae89b-1343-4a99-983c-0b79d93d9331 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:24.018107 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80aae89b-1343-4a99-983c-0b79d93d9331 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:24.038325 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-80aae89b-1343-4a99-983c-0b79d93d9331 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:23:24.039673 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-80aae89b-1343-4a99-983c-0b79d93d9331 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:23:24.100055 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 985/1957] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:23:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:23:24.109829 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71890128-25f0-4aa0-b8b0-dcd603865285 admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:24.111337 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-71890128-25f0-4aa0-b8b0-dcd603865285 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:23:24.167913 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71890128-25f0-4aa0-b8b0-dcd603865285 admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:24.171691 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ef4bb6-d7a9-4296-a3cd-80171d0a00be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0817c3320d5d41d8bba0550c8ae8dd60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:24.172302 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ef4bb6-d7a9-4296-a3cd-80171d0a00be service neutron] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:24.172582 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ef4bb6-d7a9-4296-a3cd-80171d0a00be service neutron] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:24.172884 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ef4bb6-d7a9-4296-a3cd-80171d0a00be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:24.182542 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ef4bb6-d7a9-4296-a3cd-80171d0a00be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:24.183925 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ef4bb6-d7a9-4296-a3cd-80171d0a00be service neutron] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:24.193579 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 973/1958] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:23:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:23:24.643735 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80aae89b-1343-4a99-983c-0b79d93d9331 None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:24.645495 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-80aae89b-1343-4a99-983c-0b79d93d9331 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:23:24.662739 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80aae89b-1343-4a99-983c-0b79d93d9331 None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:24.669256 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c7dfa3-95b9-4fbc-b89f-506450752ed1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:24.669908 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c7dfa3-95b9-4fbc-b89f-506450752ed1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:24.670334 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c7dfa3-95b9-4fbc-b89f-506450752ed1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:24.670893 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c7dfa3-95b9-4fbc-b89f-506450752ed1 admin admin] PATH_INFO: `/v3/projects/c0719f0fb98944edbdb5efa1691de44c` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:24.672259 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c7dfa3-95b9-4fbc-b89f-506450752ed1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c0719f0fb98944edbdb5efa1691de44c)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:24.674376 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c7dfa3-95b9-4fbc-b89f-506450752ed1 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:24.739281 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:23:24.822002 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-40c7dfa3-95b9-4fbc-b89f-506450752ed1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c0719f0fb98944edbdb5efa1691de44c', 'request_id': 'req-40c7dfa3-95b9-4fbc-b89f-506450752ed1'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:24.827812 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-40c7dfa3-95b9-4fbc-b89f-506450752ed1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c0719f0fb98944edbdb5efa1691de44c', 'request_id': 'req-40c7dfa3-95b9-4fbc-b89f-506450752ed1'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:24.927824 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-40c7dfa3-95b9-4fbc-b89f-506450752ed1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c0719f0fb98944edbdb5efa1691de44c', 'request_id': 'req-40c7dfa3-95b9-4fbc-b89f-506450752ed1'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:24.939882 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-40c7dfa3-95b9-4fbc-b89f-506450752ed1 admin admin] The token cache is being invalidate because project c0719f0fb98944edbdb5efa1691de44c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81750) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 01:23:24.940100 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-40c7dfa3-95b9-4fbc-b89f-506450752ed1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:24.940775 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-40c7dfa3-95b9-4fbc-b89f-506450752ed1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:24.942558 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 986/1959] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 01:23:24 2022] DELETE /identity/v3/projects/c0719f0fb98944edbdb5efa1691de44c => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:23:24.946412 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ef4bb6-d7a9-4296-a3cd-80171d0a00be service neutron] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:24.957196 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ef4bb6-d7a9-4296-a3cd-80171d0a00be service neutron] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:25.015062 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2ce103-148a-468d-8969-b476f8f547eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:25.015716 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2ce103-148a-468d-8969-b476f8f547eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:25.015991 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2ce103-148a-468d-8969-b476f8f547eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:25.016270 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2ce103-148a-468d-8969-b476f8f547eb admin admin] PATH_INFO: `/v3/users/132dca16d40c467682619cfad463d274` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:25.017474 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2ce103-148a-468d-8969-b476f8f547eb admin admin] RBAC: Authorizing `identity:delete_user(user_id=132dca16d40c467682619cfad463d274)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:25.019407 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2ce103-148a-468d-8969-b476f8f547eb admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:25.248953 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:23:25.263660 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-0a2ce103-148a-468d-8969-b476f8f547eb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '132dca16d40c467682619cfad463d274'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:25.326927 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-0a2ce103-148a-468d-8969-b476f8f547eb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '132dca16d40c467682619cfad463d274'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:25.327563 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-0a2ce103-148a-468d-8969-b476f8f547eb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '132dca16d40c467682619cfad463d274'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:25.335539 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-0a2ce103-148a-468d-8969-b476f8f547eb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '132dca16d40c467682619cfad463d274'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:25.336228 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-0a2ce103-148a-468d-8969-b476f8f547eb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '132dca16d40c467682619cfad463d274'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:25.366194 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 974/1960] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 01:23:24 2022] DELETE /identity/v3/users/132dca16d40c467682619cfad463d274 => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:23:25.845424 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c7dfa3-95b9-4fbc-b89f-506450752ed1 admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:25.847206 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-40c7dfa3-95b9-4fbc-b89f-506450752ed1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:23:25.858314 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2ce103-148a-468d-8969-b476f8f547eb admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:25.858314 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-0a2ce103-148a-468d-8969-b476f8f547eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:23:25.876521 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c7dfa3-95b9-4fbc-b89f-506450752ed1 admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:25.951174 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2ce103-148a-468d-8969-b476f8f547eb admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:25.953622 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4243a0df-56b8-4bd8-b042-48f95a25cb12 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5c8fc69c518412ba1aad61a9f4f558c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:25.954661 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4243a0df-56b8-4bd8-b042-48f95a25cb12 service cinder] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:25.955119 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4243a0df-56b8-4bd8-b042-48f95a25cb12 service cinder] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:25.955397 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4243a0df-56b8-4bd8-b042-48f95a25cb12 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:25.957306 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-4243a0df-56b8-4bd8-b042-48f95a25cb12 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:23:25.965873 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec4e6b6-f6d8-419e-ac02-8422038935e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:25.966445 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec4e6b6-f6d8-419e-ac02-8422038935e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:25.966730 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec4e6b6-f6d8-419e-ac02-8422038935e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:25.967007 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec4e6b6-f6d8-419e-ac02-8422038935e2 admin admin] PATH_INFO: `/v3/projects/45529684e17a43539f53e2a57142428c` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:25.968158 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec4e6b6-f6d8-419e-ac02-8422038935e2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=45529684e17a43539f53e2a57142428c)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:25.970151 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec4e6b6-f6d8-419e-ac02-8422038935e2 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:26.034572 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4243a0df-56b8-4bd8-b042-48f95a25cb12 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:26.034869 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4243a0df-56b8-4bd8-b042-48f95a25cb12 service cinder] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:26.044978 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 975/1961] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:23:25 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 01:23:26.111170 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-bec4e6b6-f6d8-419e-ac02-8422038935e2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '45529684e17a43539f53e2a57142428c', 'request_id': 'req-bec4e6b6-f6d8-419e-ac02-8422038935e2'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:26.117015 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-bec4e6b6-f6d8-419e-ac02-8422038935e2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '45529684e17a43539f53e2a57142428c', 'request_id': 'req-bec4e6b6-f6d8-419e-ac02-8422038935e2'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:26.185917 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-bec4e6b6-f6d8-419e-ac02-8422038935e2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '45529684e17a43539f53e2a57142428c', 'request_id': 'req-bec4e6b6-f6d8-419e-ac02-8422038935e2'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:26.200099 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-bec4e6b6-f6d8-419e-ac02-8422038935e2 admin admin] The token cache is being invalidate because project 45529684e17a43539f53e2a57142428c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81750) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 01:23:26.200400 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-bec4e6b6-f6d8-419e-ac02-8422038935e2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:26.201173 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-bec4e6b6-f6d8-419e-ac02-8422038935e2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:26.202981 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 987/1962] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 01:23:25 2022] DELETE /identity/v3/projects/45529684e17a43539f53e2a57142428c => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:23:26.211751 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730e550a-1276-471e-8096-0d170d132645 None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:26.212094 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730e550a-1276-471e-8096-0d170d132645 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:26.212386 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730e550a-1276-471e-8096-0d170d132645 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:26.237300 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-730e550a-1276-471e-8096-0d170d132645 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:23:26.238534 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-730e550a-1276-471e-8096-0d170d132645 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:23:26.305141 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 976/1963] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:23:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:23:26.314041 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a94fa56-d6d9-4847-bf3c-bdee537304c2 None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:26.314565 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a94fa56-d6d9-4847-bf3c-bdee537304c2 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:26.315691 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a94fa56-d6d9-4847-bf3c-bdee537304c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:26.336530 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-5a94fa56-d6d9-4847-bf3c-bdee537304c2 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:23:26.337779 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-5a94fa56-d6d9-4847-bf3c-bdee537304c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:23:26.346219 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 988/1964] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:23:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:23:26.350150 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730e550a-1276-471e-8096-0d170d132645 None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:26.359783 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730e550a-1276-471e-8096-0d170d132645 None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:26.364632 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2222dae-9268-4311-b54f-b145320963e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:26.365285 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2222dae-9268-4311-b54f-b145320963e1 admin admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:26.365597 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2222dae-9268-4311-b54f-b145320963e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:26.365871 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2222dae-9268-4311-b54f-b145320963e1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:26.366513 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2222dae-9268-4311-b54f-b145320963e1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:23:26.366881 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2222dae-9268-4311-b54f-b145320963e1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:26.368518 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2222dae-9268-4311-b54f-b145320963e1 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:26.380430 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 977/1965] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Aug 31 01:23:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:23:26.384648 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a94fa56-d6d9-4847-bf3c-bdee537304c2 None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:26.394094 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a94fa56-d6d9-4847-bf3c-bdee537304c2 None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:26.399054 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a9ff55-dcf2-4529-98e9-d590fa4df507 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:26.399659 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a9ff55-dcf2-4529-98e9-d590fa4df507 admin admin] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:26.399940 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a9ff55-dcf2-4529-98e9-d590fa4df507 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:26.400208 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a9ff55-dcf2-4529-98e9-d590fa4df507 admin admin] PATH_INFO: `/v3/projects` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:26.400942 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a9ff55-dcf2-4529-98e9-d590fa4df507 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:26.402835 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a9ff55-dcf2-4529-98e9-d590fa4df507 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:26.507621 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 989/1966] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 01:23:26 2022] POST /identity/v3/projects => generated 381 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:23:26.512195 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2222dae-9268-4311-b54f-b145320963e1 admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:26.524242 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2222dae-9268-4311-b54f-b145320963e1 admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:26.591234 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bfeb3e-533f-4883-9a51-9a6b6b92cf5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:26.591623 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bfeb3e-533f-4883-9a51-9a6b6b92cf5c admin admin] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:26.591925 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bfeb3e-533f-4883-9a51-9a6b6b92cf5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:26.592206 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bfeb3e-533f-4883-9a51-9a6b6b92cf5c admin admin] PATH_INFO: `/v3/users` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:26.592932 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bfeb3e-533f-4883-9a51-9a6b6b92cf5c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:26.595045 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bfeb3e-533f-4883-9a51-9a6b6b92cf5c admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:26.703089 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 978/1967] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 01:23:26 2022] POST /identity/v3/users => generated 291 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:23:26.707237 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a9ff55-dcf2-4529-98e9-d590fa4df507 admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:26.719754 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a9ff55-dcf2-4529-98e9-d590fa4df507 admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:26.730327 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef5acfb-e6c7-4789-a3c7-26236578880d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:26.731313 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef5acfb-e6c7-4789-a3c7-26236578880d admin admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:26.731801 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef5acfb-e6c7-4789-a3c7-26236578880d admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:26.732267 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef5acfb-e6c7-4789-a3c7-26236578880d admin admin] PATH_INFO: `/v3/roles` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:26.733138 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef5acfb-e6c7-4789-a3c7-26236578880d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:26.735053 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef5acfb-e6c7-4789-a3c7-26236578880d admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:26.746923 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 990/1968] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:23:26 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:23:26.751215 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bfeb3e-533f-4883-9a51-9a6b6b92cf5c admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:26.763452 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bfeb3e-533f-4883-9a51-9a6b6b92cf5c admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:26.772607 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68254277-7844-47a0-8126-03841e2f71a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:26.773392 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68254277-7844-47a0-8126-03841e2f71a7 admin admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:26.773682 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68254277-7844-47a0-8126-03841e2f71a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:26.773962 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68254277-7844-47a0-8126-03841e2f71a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:26.774871 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68254277-7844-47a0-8126-03841e2f71a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:26.776581 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68254277-7844-47a0-8126-03841e2f71a7 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:26.789242 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 979/1969] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:23:26 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:23:26.793779 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef5acfb-e6c7-4789-a3c7-26236578880d admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:26.806335 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef5acfb-e6c7-4789-a3c7-26236578880d admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:26.811456 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6885381-0fbf-4b5a-9f4a-194f9d6537a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:26.812390 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6885381-0fbf-4b5a-9f4a-194f9d6537a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:26.812673 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6885381-0fbf-4b5a-9f4a-194f9d6537a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:26.812986 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6885381-0fbf-4b5a-9f4a-194f9d6537a5 admin admin] PATH_INFO: `/v3/projects/df0fcbb4574b4acca38369b240444848/users/56212024c03448b780095bc58bd5a087/roles/d0b9768f4d014766b9326387f2e11f34` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:26.830257 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6885381-0fbf-4b5a-9f4a-194f9d6537a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=df0fcbb4574b4acca38369b240444848, user_id=56212024c03448b780095bc58bd5a087, role_id=d0b9768f4d014766b9326387f2e11f34)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:26.836295 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6885381-0fbf-4b5a-9f4a-194f9d6537a5 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:26.925677 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 991/1970] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 01:23:26 2022] PUT /identity/v3/projects/df0fcbb4574b4acca38369b240444848/users/56212024c03448b780095bc58bd5a087/roles/d0b9768f4d014766b9326387f2e11f34 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 01:23:26.936642 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a91310c-b129-47cb-87e1-b0ffa0726b92 None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:26.937097 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a91310c-b129-47cb-87e1-b0ffa0726b92 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:26.939022 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a91310c-b129-47cb-87e1-b0ffa0726b92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:26.964792 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-9a91310c-b129-47cb-87e1-b0ffa0726b92 None None] MFA Rules not processed for user `56212024c03448b780095bc58bd5a087`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:23:26.965974 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-9a91310c-b129-47cb-87e1-b0ffa0726b92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:23:26.979932 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:23:27.030463 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 980/1971] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:23:26 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:23:27.034914 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6885381-0fbf-4b5a-9f4a-194f9d6537a5 admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:27.095890 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6885381-0fbf-4b5a-9f4a-194f9d6537a5 admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:27.097872 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:23:27.103643 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75ce1ab-5b1b-4d2b-b5f6-ccad87cbaadc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:27.103896 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75ce1ab-5b1b-4d2b-b5f6-ccad87cbaadc admin admin] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:27.104222 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75ce1ab-5b1b-4d2b-b5f6-ccad87cbaadc admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:27.104500 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75ce1ab-5b1b-4d2b-b5f6-ccad87cbaadc admin admin] PATH_INFO: `/v3/projects` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:27.105180 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75ce1ab-5b1b-4d2b-b5f6-ccad87cbaadc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:27.107107 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75ce1ab-5b1b-4d2b-b5f6-ccad87cbaadc admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:27.185342 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 992/1972] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 01:23:27 2022] POST /identity/v3/projects => generated 381 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:23:27.186537 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a91310c-b129-47cb-87e1-b0ffa0726b92 None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:27.198035 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a91310c-b129-47cb-87e1-b0ffa0726b92 None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:27.259068 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d097898f-5959-4a5b-bb74-0239fc7bdcb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:27.259682 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d097898f-5959-4a5b-bb74-0239fc7bdcb3 admin admin] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:27.259963 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d097898f-5959-4a5b-bb74-0239fc7bdcb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:27.260265 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d097898f-5959-4a5b-bb74-0239fc7bdcb3 admin admin] PATH_INFO: `/v3/users` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:27.260931 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d097898f-5959-4a5b-bb74-0239fc7bdcb3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:27.262908 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d097898f-5959-4a5b-bb74-0239fc7bdcb3 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:27.372173 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 981/1973] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 01:23:27 2022] POST /identity/v3/users => generated 297 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:23:27.373535 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75ce1ab-5b1b-4d2b-b5f6-ccad87cbaadc admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:27.384119 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75ce1ab-5b1b-4d2b-b5f6-ccad87cbaadc admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:27.389075 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656800cb-749f-4080-be13-ff2a9de9d37d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:27.389692 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656800cb-749f-4080-be13-ff2a9de9d37d admin admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:27.396760 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656800cb-749f-4080-be13-ff2a9de9d37d admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:27.396760 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656800cb-749f-4080-be13-ff2a9de9d37d admin admin] PATH_INFO: `/v3/roles` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:27.396760 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656800cb-749f-4080-be13-ff2a9de9d37d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:27.396760 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656800cb-749f-4080-be13-ff2a9de9d37d admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:27.404998 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 993/1974] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:23:27 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:23:27.409415 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d097898f-5959-4a5b-bb74-0239fc7bdcb3 admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:27.420314 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d097898f-5959-4a5b-bb74-0239fc7bdcb3 admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:27.425057 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43403ad2-8f1a-4a19-8ccb-bf910831b3a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:27.425716 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43403ad2-8f1a-4a19-8ccb-bf910831b3a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:27.426000 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43403ad2-8f1a-4a19-8ccb-bf910831b3a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:27.426278 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43403ad2-8f1a-4a19-8ccb-bf910831b3a7 admin admin] PATH_INFO: `/v3/projects/0cd95857a3b649ceb4af7b485497b4e0/users/d5cd148670ca406aa4e4a3a50fe8bc89/roles/d009411496b8452b88e77ffb8db015a7` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:27.441159 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43403ad2-8f1a-4a19-8ccb-bf910831b3a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0cd95857a3b649ceb4af7b485497b4e0, user_id=d5cd148670ca406aa4e4a3a50fe8bc89, role_id=d009411496b8452b88e77ffb8db015a7)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:27.443876 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43403ad2-8f1a-4a19-8ccb-bf910831b3a7 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:27.500178 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 982/1975] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 01:23:27 2022] PUT /identity/v3/projects/0cd95857a3b649ceb4af7b485497b4e0/users/d5cd148670ca406aa4e4a3a50fe8bc89/roles/d009411496b8452b88e77ffb8db015a7 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:23:27.504350 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656800cb-749f-4080-be13-ff2a9de9d37d admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:27.514877 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656800cb-749f-4080-be13-ff2a9de9d37d admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:27.570055 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c748de-54e4-4f18-92c0-46c9ed737564 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:27.570655 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c748de-54e4-4f18-92c0-46c9ed737564 admin admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:27.570934 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c748de-54e4-4f18-92c0-46c9ed737564 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:27.571204 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c748de-54e4-4f18-92c0-46c9ed737564 admin admin] PATH_INFO: `/v3/roles` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:27.571828 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c748de-54e4-4f18-92c0-46c9ed737564 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:27.573555 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c748de-54e4-4f18-92c0-46c9ed737564 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:27.583842 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 994/1976] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:23:27 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:23:27.587953 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43403ad2-8f1a-4a19-8ccb-bf910831b3a7 admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:27.598114 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43403ad2-8f1a-4a19-8ccb-bf910831b3a7 admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:27.603343 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4418dc0-1904-475e-baa5-6eb9e76cc3bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:27.604114 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4418dc0-1904-475e-baa5-6eb9e76cc3bf admin admin] REQUEST_METHOD: `PUT` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:27.604395 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4418dc0-1904-475e-baa5-6eb9e76cc3bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:27.604666 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4418dc0-1904-475e-baa5-6eb9e76cc3bf admin admin] PATH_INFO: `/v3/projects/0cd95857a3b649ceb4af7b485497b4e0/users/d5cd148670ca406aa4e4a3a50fe8bc89/roles/d009411496b8452b88e77ffb8db015a7` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:27.606712 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4418dc0-1904-475e-baa5-6eb9e76cc3bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=0cd95857a3b649ceb4af7b485497b4e0, user_id=d5cd148670ca406aa4e4a3a50fe8bc89, role_id=d009411496b8452b88e77ffb8db015a7)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:27.609402 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4418dc0-1904-475e-baa5-6eb9e76cc3bf admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:27.619765 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 983/1977] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 01:23:27 2022] PUT /identity/v3/projects/0cd95857a3b649ceb4af7b485497b4e0/users/d5cd148670ca406aa4e4a3a50fe8bc89/roles/d009411496b8452b88e77ffb8db015a7 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:23:27.636353 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56cddbc-a9fc-490d-b3d2-9c16f1cf2565 None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:27.636676 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56cddbc-a9fc-490d-b3d2-9c16f1cf2565 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:27.636989 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56cddbc-a9fc-490d-b3d2-9c16f1cf2565 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:27.666724 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-e56cddbc-a9fc-490d-b3d2-9c16f1cf2565 None None] MFA Rules not processed for user `d5cd148670ca406aa4e4a3a50fe8bc89`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:23:27.668059 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-e56cddbc-a9fc-490d-b3d2-9c16f1cf2565 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:23:27.742597 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 995/1978] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:23:27 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:23:27.976233 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4418dc0-1904-475e-baa5-6eb9e76cc3bf admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:27.982401 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-b4418dc0-1904-475e-baa5-6eb9e76cc3bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:23:28.043239 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4418dc0-1904-475e-baa5-6eb9e76cc3bf admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:28.046807 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d77320-9945-40bd-9be3-1fd4e6590755 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5c8fc69c518412ba1aad61a9f4f558c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:28.074122 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d77320-9945-40bd-9be3-1fd4e6590755 service cinder] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:28.074122 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d77320-9945-40bd-9be3-1fd4e6590755 service cinder] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:28.074122 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d77320-9945-40bd-9be3-1fd4e6590755 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:28.074122 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d77320-9945-40bd-9be3-1fd4e6590755 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:28.074122 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d77320-9945-40bd-9be3-1fd4e6590755 service cinder] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:28.074122 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 984/1979] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:23:27 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:23:28.122787 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:23:28.241090 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56cddbc-a9fc-490d-b3d2-9c16f1cf2565 None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:28.251307 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56cddbc-a9fc-490d-b3d2-9c16f1cf2565 None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:28.254958 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab33bfc-c5ff-4e45-9f90-4561aecde047 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5c8fc69c518412ba1aad61a9f4f558c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:28.255796 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab33bfc-c5ff-4e45-9f90-4561aecde047 service cinder] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:28.255796 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab33bfc-c5ff-4e45-9f90-4561aecde047 service cinder] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:28.256194 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab33bfc-c5ff-4e45-9f90-4561aecde047 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:28.325897 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab33bfc-c5ff-4e45-9f90-4561aecde047 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:28.327539 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab33bfc-c5ff-4e45-9f90-4561aecde047 service cinder] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:28.339952 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 996/1980] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:23:28 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:23:28.567718 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:23:28.867441 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d77320-9945-40bd-9be3-1fd4e6590755 service cinder] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:28.929696 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d77320-9945-40bd-9be3-1fd4e6590755 service cinder] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:28.934188 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45ef1ab-2d22-47a2-9c8f-bea66c649122 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:28.936836 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45ef1ab-2d22-47a2-9c8f-bea66c649122 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:28.936836 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45ef1ab-2d22-47a2-9c8f-bea66c649122 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:28.936836 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45ef1ab-2d22-47a2-9c8f-bea66c649122 admin admin] PATH_INFO: `/v3/users/56212024c03448b780095bc58bd5a087` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:28.936836 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45ef1ab-2d22-47a2-9c8f-bea66c649122 admin admin] RBAC: Authorizing `identity:delete_user(user_id=56212024c03448b780095bc58bd5a087)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:28.938388 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45ef1ab-2d22-47a2-9c8f-bea66c649122 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:29.054420 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=78,keystone:DELETE=17,keystone:INSERT=5 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:23:29.113271 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-d45ef1ab-2d22-47a2-9c8f-bea66c649122 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '56212024c03448b780095bc58bd5a087'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:29.191776 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-d45ef1ab-2d22-47a2-9c8f-bea66c649122 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '56212024c03448b780095bc58bd5a087'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:29.192433 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-d45ef1ab-2d22-47a2-9c8f-bea66c649122 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '56212024c03448b780095bc58bd5a087'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:29.199955 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-d45ef1ab-2d22-47a2-9c8f-bea66c649122 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '56212024c03448b780095bc58bd5a087'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:29.200647 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-d45ef1ab-2d22-47a2-9c8f-bea66c649122 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '56212024c03448b780095bc58bd5a087'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:29.217684 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 985/1981] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 01:23:28 2022] DELETE /identity/v3/users/56212024c03448b780095bc58bd5a087 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:23:29.222683 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d15a19f-7f2a-4b12-93e7-83807ab310c2 None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:29.222958 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d15a19f-7f2a-4b12-93e7-83807ab310c2 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:29.223232 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d15a19f-7f2a-4b12-93e7-83807ab310c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:29.244879 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-4d15a19f-7f2a-4b12-93e7-83807ab310c2 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:23:29.246125 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-4d15a19f-7f2a-4b12-93e7-83807ab310c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:23:29.304709 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 997/1982] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:23:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:23:29.314702 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45ef1ab-2d22-47a2-9c8f-bea66c649122 admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:29.316175 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-d45ef1ab-2d22-47a2-9c8f-bea66c649122 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:23:29.371391 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:23:29.376154 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45ef1ab-2d22-47a2-9c8f-bea66c649122 admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:29.379933 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f55207-1fba-46d5-8fc7-20fbd6a6dc9e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0817c3320d5d41d8bba0550c8ae8dd60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:29.380557 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f55207-1fba-46d5-8fc7-20fbd6a6dc9e service neutron] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:29.380893 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f55207-1fba-46d5-8fc7-20fbd6a6dc9e service neutron] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:29.381176 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f55207-1fba-46d5-8fc7-20fbd6a6dc9e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:29.390862 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f55207-1fba-46d5-8fc7-20fbd6a6dc9e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:29.392348 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f55207-1fba-46d5-8fc7-20fbd6a6dc9e service neutron] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:29.402876 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 986/1983] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:23:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:23:29.740861 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:23:29.837310 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d15a19f-7f2a-4b12-93e7-83807ab310c2 None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:29.838757 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-4d15a19f-7f2a-4b12-93e7-83807ab310c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:23:29.850905 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d15a19f-7f2a-4b12-93e7-83807ab310c2 None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:29.855982 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a70d946-acc8-4afd-bec4-6f77a7e737f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:29.856621 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a70d946-acc8-4afd-bec4-6f77a7e737f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:29.856941 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a70d946-acc8-4afd-bec4-6f77a7e737f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:29.857218 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a70d946-acc8-4afd-bec4-6f77a7e737f0 admin admin] PATH_INFO: `/v3/projects/df0fcbb4574b4acca38369b240444848` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:29.858350 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a70d946-acc8-4afd-bec4-6f77a7e737f0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=df0fcbb4574b4acca38369b240444848)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:29.860263 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a70d946-acc8-4afd-bec4-6f77a7e737f0 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:29.953443 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-4a70d946-acc8-4afd-bec4-6f77a7e737f0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'df0fcbb4574b4acca38369b240444848', 'request_id': 'req-4a70d946-acc8-4afd-bec4-6f77a7e737f0'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:29.959369 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-4a70d946-acc8-4afd-bec4-6f77a7e737f0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'df0fcbb4574b4acca38369b240444848', 'request_id': 'req-4a70d946-acc8-4afd-bec4-6f77a7e737f0'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:30.035308 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:23:30.059284 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-4a70d946-acc8-4afd-bec4-6f77a7e737f0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'df0fcbb4574b4acca38369b240444848', 'request_id': 'req-4a70d946-acc8-4afd-bec4-6f77a7e737f0'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:30.071685 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-4a70d946-acc8-4afd-bec4-6f77a7e737f0 admin admin] The token cache is being invalidate because project df0fcbb4574b4acca38369b240444848 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81750) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 01:23:30.071986 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-4a70d946-acc8-4afd-bec4-6f77a7e737f0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:30.072650 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-4a70d946-acc8-4afd-bec4-6f77a7e737f0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:30.076672 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 998/1984] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 01:23:29 2022] DELETE /identity/v3/projects/df0fcbb4574b4acca38369b240444848 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:23:30.082947 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f55207-1fba-46d5-8fc7-20fbd6a6dc9e service neutron] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:30.092922 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f55207-1fba-46d5-8fc7-20fbd6a6dc9e service neutron] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:30.104519 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:23:30.155171 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d868ba7-a5a5-49e0-a4fd-448d0ebe3a8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:30.155861 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d868ba7-a5a5-49e0-a4fd-448d0ebe3a8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:30.156127 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d868ba7-a5a5-49e0-a4fd-448d0ebe3a8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:30.156408 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d868ba7-a5a5-49e0-a4fd-448d0ebe3a8f admin admin] PATH_INFO: `/v3/users/d5cd148670ca406aa4e4a3a50fe8bc89` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:30.157572 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d868ba7-a5a5-49e0-a4fd-448d0ebe3a8f admin admin] RBAC: Authorizing `identity:delete_user(user_id=d5cd148670ca406aa4e4a3a50fe8bc89)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:30.159568 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d868ba7-a5a5-49e0-a4fd-448d0ebe3a8f admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:30.350123 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-3d868ba7-a5a5-49e0-a4fd-448d0ebe3a8f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd5cd148670ca406aa4e4a3a50fe8bc89'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:30.425411 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-3d868ba7-a5a5-49e0-a4fd-448d0ebe3a8f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd5cd148670ca406aa4e4a3a50fe8bc89'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:30.426047 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-3d868ba7-a5a5-49e0-a4fd-448d0ebe3a8f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd5cd148670ca406aa4e4a3a50fe8bc89'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:30.433434 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-3d868ba7-a5a5-49e0-a4fd-448d0ebe3a8f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd5cd148670ca406aa4e4a3a50fe8bc89'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:30.434063 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-3d868ba7-a5a5-49e0-a4fd-448d0ebe3a8f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd5cd148670ca406aa4e4a3a50fe8bc89'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:30.451534 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 987/1985] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 01:23:30 2022] DELETE /identity/v3/users/d5cd148670ca406aa4e4a3a50fe8bc89 => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:23:30.506272 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:23:30.660967 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:23:30.948742 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a70d946-acc8-4afd-bec4-6f77a7e737f0 admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:30.948742 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-4a70d946-acc8-4afd-bec4-6f77a7e737f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:23:30.962247 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a70d946-acc8-4afd-bec4-6f77a7e737f0 admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:31.022746 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c157d81-4564-4551-90b6-602db4cba2b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:31.023383 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c157d81-4564-4551-90b6-602db4cba2b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:31.023674 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c157d81-4564-4551-90b6-602db4cba2b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:31.023956 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c157d81-4564-4551-90b6-602db4cba2b0 admin admin] PATH_INFO: `/v3/projects/0cd95857a3b649ceb4af7b485497b4e0` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:31.025152 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c157d81-4564-4551-90b6-602db4cba2b0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0cd95857a3b649ceb4af7b485497b4e0)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:31.027053 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c157d81-4564-4551-90b6-602db4cba2b0 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:31.138563 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:23:31.155759 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-2c157d81-4564-4551-90b6-602db4cba2b0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0cd95857a3b649ceb4af7b485497b4e0', 'request_id': 'req-2c157d81-4564-4551-90b6-602db4cba2b0'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:31.161935 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-2c157d81-4564-4551-90b6-602db4cba2b0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0cd95857a3b649ceb4af7b485497b4e0', 'request_id': 'req-2c157d81-4564-4551-90b6-602db4cba2b0'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:31.241071 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-2c157d81-4564-4551-90b6-602db4cba2b0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0cd95857a3b649ceb4af7b485497b4e0', 'request_id': 'req-2c157d81-4564-4551-90b6-602db4cba2b0'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:31.251627 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-2c157d81-4564-4551-90b6-602db4cba2b0 admin admin] The token cache is being invalidate because project 0cd95857a3b649ceb4af7b485497b4e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81750) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 01:23:31.252015 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-2c157d81-4564-4551-90b6-602db4cba2b0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:31.252812 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-2c157d81-4564-4551-90b6-602db4cba2b0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:31.255548 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 999/1986] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 01:23:30 2022] DELETE /identity/v3/projects/0cd95857a3b649ceb4af7b485497b4e0 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:23:31.260389 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebe2811-7489-4f97-9485-b87a52635cc2 None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:31.263450 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebe2811-7489-4f97-9485-b87a52635cc2 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:31.263450 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebe2811-7489-4f97-9485-b87a52635cc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:31.282600 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:23:31.284347 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-bebe2811-7489-4f97-9485-b87a52635cc2 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:23:31.290080 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-bebe2811-7489-4f97-9485-b87a52635cc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:23:31.351596 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 988/1987] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:23:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:23:31.360402 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e35050-dc75-4bad-b923-670e606e321e None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:31.362116 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e35050-dc75-4bad-b923-670e606e321e None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:31.362642 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e35050-dc75-4bad-b923-670e606e321e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:31.385418 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-e7e35050-dc75-4bad-b923-670e606e321e None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:23:31.386651 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-e7e35050-dc75-4bad-b923-670e606e321e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:23:31.396147 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1000/1988] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:23:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:23:31.400247 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebe2811-7489-4f97-9485-b87a52635cc2 None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:31.410497 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebe2811-7489-4f97-9485-b87a52635cc2 None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:31.415538 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab2cd02-43d9-447b-884e-40f8ed3638d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:31.416212 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab2cd02-43d9-447b-884e-40f8ed3638d9 admin admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:31.416501 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab2cd02-43d9-447b-884e-40f8ed3638d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:31.416849 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab2cd02-43d9-447b-884e-40f8ed3638d9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:31.417505 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab2cd02-43d9-447b-884e-40f8ed3638d9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:23:31.417815 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab2cd02-43d9-447b-884e-40f8ed3638d9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:31.419429 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab2cd02-43d9-447b-884e-40f8ed3638d9 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:31.431126 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 989/1989] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Aug 31 01:23:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:23:31.436346 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e35050-dc75-4bad-b923-670e606e321e None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:31.447048 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e35050-dc75-4bad-b923-670e606e321e None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:31.451934 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51984163-4a0a-4b3b-b0da-59569395b757 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:31.452521 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51984163-4a0a-4b3b-b0da-59569395b757 admin admin] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:31.452877 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51984163-4a0a-4b3b-b0da-59569395b757 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:31.453154 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51984163-4a0a-4b3b-b0da-59569395b757 admin admin] PATH_INFO: `/v3/projects` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:31.453824 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51984163-4a0a-4b3b-b0da-59569395b757 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:31.455975 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51984163-4a0a-4b3b-b0da-59569395b757 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:31.604498 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1001/1990] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 01:23:31 2022] POST /identity/v3/projects => generated 385 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:23:31.609176 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab2cd02-43d9-447b-884e-40f8ed3638d9 admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:31.619180 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab2cd02-43d9-447b-884e-40f8ed3638d9 admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:31.694788 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88e4141-2bd3-4961-8b0d-eabf2c29a793 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:31.696419 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88e4141-2bd3-4961-8b0d-eabf2c29a793 admin admin] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:31.696419 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88e4141-2bd3-4961-8b0d-eabf2c29a793 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:31.696419 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88e4141-2bd3-4961-8b0d-eabf2c29a793 admin admin] PATH_INFO: `/v3/users` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:31.696714 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88e4141-2bd3-4961-8b0d-eabf2c29a793 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:31.698651 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88e4141-2bd3-4961-8b0d-eabf2c29a793 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:31.815746 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 990/1991] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 01:23:31 2022] POST /identity/v3/users => generated 293 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:23:31.817167 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51984163-4a0a-4b3b-b0da-59569395b757 admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:31.828102 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51984163-4a0a-4b3b-b0da-59569395b757 admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:31.838048 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0942dd-c44a-48f9-a2c5-dceecb63fa21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:31.838048 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0942dd-c44a-48f9-a2c5-dceecb63fa21 admin admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:31.838048 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0942dd-c44a-48f9-a2c5-dceecb63fa21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:31.838048 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0942dd-c44a-48f9-a2c5-dceecb63fa21 admin admin] PATH_INFO: `/v3/roles` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:31.838380 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0942dd-c44a-48f9-a2c5-dceecb63fa21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:31.838380 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0942dd-c44a-48f9-a2c5-dceecb63fa21 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:31.846354 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1002/1992] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:23:31 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:23:31.850497 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88e4141-2bd3-4961-8b0d-eabf2c29a793 admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:31.866053 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88e4141-2bd3-4961-8b0d-eabf2c29a793 admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:31.871367 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046c913d-5f92-4870-945d-430f9b260141 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:31.871973 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046c913d-5f92-4870-945d-430f9b260141 admin admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:31.872418 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046c913d-5f92-4870-945d-430f9b260141 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:31.872694 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046c913d-5f92-4870-945d-430f9b260141 admin admin] PATH_INFO: `/v3/roles` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:31.873381 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046c913d-5f92-4870-945d-430f9b260141 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:31.875301 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046c913d-5f92-4870-945d-430f9b260141 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:31.888249 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 991/1993] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:23:31 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:23:31.891968 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0942dd-c44a-48f9-a2c5-dceecb63fa21 admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:31.912323 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0942dd-c44a-48f9-a2c5-dceecb63fa21 admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:31.918258 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1735601a-805a-46b3-a036-d5ff5ac68b41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:31.919042 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1735601a-805a-46b3-a036-d5ff5ac68b41 admin admin] REQUEST_METHOD: `PUT` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:31.924783 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1735601a-805a-46b3-a036-d5ff5ac68b41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:31.924783 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1735601a-805a-46b3-a036-d5ff5ac68b41 admin admin] PATH_INFO: `/v3/projects/64a2333f399f41d693016c4fd9f41af5/users/5055de19b9b34875809f37665612cdfb/roles/d0b9768f4d014766b9326387f2e11f34` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:31.937577 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1735601a-805a-46b3-a036-d5ff5ac68b41 admin admin] RBAC: Authorizing `identity:create_grant(project_id=64a2333f399f41d693016c4fd9f41af5, user_id=5055de19b9b34875809f37665612cdfb, role_id=d0b9768f4d014766b9326387f2e11f34)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:31.940348 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1735601a-805a-46b3-a036-d5ff5ac68b41 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:32.062479 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1003/1994] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 01:23:31 2022] PUT /identity/v3/projects/64a2333f399f41d693016c4fd9f41af5/users/5055de19b9b34875809f37665612cdfb/roles/d0b9768f4d014766b9326387f2e11f34 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:23:32.070716 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b86f7b4-ac7c-44d4-a874-54353e863eb2 None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:32.071142 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b86f7b4-ac7c-44d4-a874-54353e863eb2 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:32.071494 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b86f7b4-ac7c-44d4-a874-54353e863eb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:32.077280 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:23:32.101314 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-1b86f7b4-ac7c-44d4-a874-54353e863eb2 None None] MFA Rules not processed for user `5055de19b9b34875809f37665612cdfb`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:23:32.102597 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-1b86f7b4-ac7c-44d4-a874-54353e863eb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:23:32.174131 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 992/1995] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:23:32 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:23:32.178587 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1735601a-805a-46b3-a036-d5ff5ac68b41 admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:32.235481 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:23:32.240427 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1735601a-805a-46b3-a036-d5ff5ac68b41 admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:32.245079 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3186e0-5e82-48a9-8061-58501d09f34a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:32.245682 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3186e0-5e82-48a9-8061-58501d09f34a admin admin] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:32.250817 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3186e0-5e82-48a9-8061-58501d09f34a admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:32.250817 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3186e0-5e82-48a9-8061-58501d09f34a admin admin] PATH_INFO: `/v3/projects` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:32.250817 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3186e0-5e82-48a9-8061-58501d09f34a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:32.250817 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3186e0-5e82-48a9-8061-58501d09f34a admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:32.323831 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1004/1996] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 01:23:32 2022] POST /identity/v3/projects => generated 385 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:23:32.327808 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b86f7b4-ac7c-44d4-a874-54353e863eb2 None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:32.340586 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b86f7b4-ac7c-44d4-a874-54353e863eb2 None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:32.396845 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d969b9bb-bca0-4a0b-9ba8-d20d1388e8cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:32.397452 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d969b9bb-bca0-4a0b-9ba8-d20d1388e8cb admin admin] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:32.397742 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d969b9bb-bca0-4a0b-9ba8-d20d1388e8cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:32.397999 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d969b9bb-bca0-4a0b-9ba8-d20d1388e8cb admin admin] PATH_INFO: `/v3/users` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:32.398626 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d969b9bb-bca0-4a0b-9ba8-d20d1388e8cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:32.400533 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d969b9bb-bca0-4a0b-9ba8-d20d1388e8cb admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:32.511806 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 993/1997] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 01:23:32 2022] POST /identity/v3/users => generated 299 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:23:32.515675 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3186e0-5e82-48a9-8061-58501d09f34a admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:32.526231 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3186e0-5e82-48a9-8061-58501d09f34a admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:32.530875 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7194beaa-141b-4e5c-a67d-0a6f663ee64b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:32.531504 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7194beaa-141b-4e5c-a67d-0a6f663ee64b admin admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:32.531786 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7194beaa-141b-4e5c-a67d-0a6f663ee64b admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:32.532061 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7194beaa-141b-4e5c-a67d-0a6f663ee64b admin admin] PATH_INFO: `/v3/roles` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:32.532710 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7194beaa-141b-4e5c-a67d-0a6f663ee64b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:32.534323 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7194beaa-141b-4e5c-a67d-0a6f663ee64b admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:32.544962 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1005/1998] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:23:32 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:23:32.548945 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d969b9bb-bca0-4a0b-9ba8-d20d1388e8cb admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:32.559407 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d969b9bb-bca0-4a0b-9ba8-d20d1388e8cb admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:32.561373 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:23:32.564875 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53ed08a-7055-471b-8e68-992967516c1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:32.565749 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53ed08a-7055-471b-8e68-992967516c1b admin admin] REQUEST_METHOD: `PUT` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:32.566033 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53ed08a-7055-471b-8e68-992967516c1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:32.566310 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53ed08a-7055-471b-8e68-992967516c1b admin admin] PATH_INFO: `/v3/projects/c9ef310bc8e046d9bea9e36bc6573c72/users/3067b984964e41c8ae3f74fb89783241/roles/d009411496b8452b88e77ffb8db015a7` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:32.584193 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53ed08a-7055-471b-8e68-992967516c1b admin admin] RBAC: Authorizing `identity:create_grant(project_id=c9ef310bc8e046d9bea9e36bc6573c72, user_id=3067b984964e41c8ae3f74fb89783241, role_id=d009411496b8452b88e77ffb8db015a7)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:32.586956 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53ed08a-7055-471b-8e68-992967516c1b admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:32.625466 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:23:32.683814 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 994/1999] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 01:23:32 2022] PUT /identity/v3/projects/c9ef310bc8e046d9bea9e36bc6573c72/users/3067b984964e41c8ae3f74fb89783241/roles/d009411496b8452b88e77ffb8db015a7 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:23:32.688079 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7194beaa-141b-4e5c-a67d-0a6f663ee64b admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:32.705838 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7194beaa-141b-4e5c-a67d-0a6f663ee64b admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:32.778821 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467fcc61-7d66-411c-8dba-a50de48c2ff3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:32.779456 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467fcc61-7d66-411c-8dba-a50de48c2ff3 admin admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:32.779740 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467fcc61-7d66-411c-8dba-a50de48c2ff3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:32.780017 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467fcc61-7d66-411c-8dba-a50de48c2ff3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:32.780661 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467fcc61-7d66-411c-8dba-a50de48c2ff3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:32.782311 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467fcc61-7d66-411c-8dba-a50de48c2ff3 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:32.792614 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1006/2000] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:23:32 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 01:23:32.796938 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53ed08a-7055-471b-8e68-992967516c1b admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:32.810345 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53ed08a-7055-471b-8e68-992967516c1b admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:32.815620 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7d26a0-7a38-4ab7-8d8f-14d311122df7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:32.816294 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7d26a0-7a38-4ab7-8d8f-14d311122df7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:32.816573 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7d26a0-7a38-4ab7-8d8f-14d311122df7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:32.816887 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7d26a0-7a38-4ab7-8d8f-14d311122df7 admin admin] PATH_INFO: `/v3/projects/c9ef310bc8e046d9bea9e36bc6573c72/users/3067b984964e41c8ae3f74fb89783241/roles/d009411496b8452b88e77ffb8db015a7` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:32.819667 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7d26a0-7a38-4ab7-8d8f-14d311122df7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c9ef310bc8e046d9bea9e36bc6573c72, user_id=3067b984964e41c8ae3f74fb89783241, role_id=d009411496b8452b88e77ffb8db015a7)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:32.822587 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7d26a0-7a38-4ab7-8d8f-14d311122df7 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:32.843473 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:23:32.846927 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 995/2001] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 01:23:32 2022] PUT /identity/v3/projects/c9ef310bc8e046d9bea9e36bc6573c72/users/3067b984964e41c8ae3f74fb89783241/roles/d009411496b8452b88e77ffb8db015a7 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:23:32.856927 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd27d76c-a6fb-4ae4-8e0b-b59b917964c2 None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:32.857452 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd27d76c-a6fb-4ae4-8e0b-b59b917964c2 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:32.857786 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd27d76c-a6fb-4ae4-8e0b-b59b917964c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:32.889194 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-fd27d76c-a6fb-4ae4-8e0b-b59b917964c2 None None] MFA Rules not processed for user `3067b984964e41c8ae3f74fb89783241`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:23:32.890438 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-fd27d76c-a6fb-4ae4-8e0b-b59b917964c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:23:32.923217 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:23:32.966293 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1007/2002] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:23:32 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:23:32.978185 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7d26a0-7a38-4ab7-8d8f-14d311122df7 admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:32.980692 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-5b7d26a0-7a38-4ab7-8d8f-14d311122df7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:23:33.048238 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7d26a0-7a38-4ab7-8d8f-14d311122df7 admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:33.054432 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c44db7-fb33-4aeb-b369-177796f4559d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5c8fc69c518412ba1aad61a9f4f558c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:33.055347 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c44db7-fb33-4aeb-b369-177796f4559d service cinder] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:33.055705 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c44db7-fb33-4aeb-b369-177796f4559d service cinder] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:33.056047 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c44db7-fb33-4aeb-b369-177796f4559d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:33.122870 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:23:33.128121 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c44db7-fb33-4aeb-b369-177796f4559d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:33.129797 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c44db7-fb33-4aeb-b369-177796f4559d service cinder] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:33.139254 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 996/2003] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:23:32 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:23:33.191722 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:23:33.404035 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:23:33.683225 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:23:34.202273 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:INSERT=4,keystone:DELETE=8,keystone:SELECT=88 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:23:34.448410 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:23:35.391482 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:23:35.599070 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:23:35.720229 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd27d76c-a6fb-4ae4-8e0b-b59b917964c2 None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:35.722761 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-fd27d76c-a6fb-4ae4-8e0b-b59b917964c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:23:35.828000 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd27d76c-a6fb-4ae4-8e0b-b59b917964c2 None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:35.833013 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d352b6-28f8-479c-95b3-135fe4eb7665 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:35.833659 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d352b6-28f8-479c-95b3-135fe4eb7665 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:35.833942 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d352b6-28f8-479c-95b3-135fe4eb7665 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:35.834222 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d352b6-28f8-479c-95b3-135fe4eb7665 admin admin] PATH_INFO: `/v3/users/016f7b40b0ce4c8297e65d943a65da4d` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:35.835477 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d352b6-28f8-479c-95b3-135fe4eb7665 admin admin] RBAC: Authorizing `identity:delete_user(user_id=016f7b40b0ce4c8297e65d943a65da4d)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:35.837367 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d352b6-28f8-479c-95b3-135fe4eb7665 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:36.153716 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-27d352b6-28f8-479c-95b3-135fe4eb7665 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '016f7b40b0ce4c8297e65d943a65da4d'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:36.168633 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-27d352b6-28f8-479c-95b3-135fe4eb7665 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '016f7b40b0ce4c8297e65d943a65da4d'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:36.170095 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-27d352b6-28f8-479c-95b3-135fe4eb7665 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '016f7b40b0ce4c8297e65d943a65da4d'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:36.181503 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-27d352b6-28f8-479c-95b3-135fe4eb7665 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '016f7b40b0ce4c8297e65d943a65da4d'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:36.288909 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-27d352b6-28f8-479c-95b3-135fe4eb7665 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '016f7b40b0ce4c8297e65d943a65da4d'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:36.296800 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1008/2004] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 01:23:35 2022] DELETE /identity/v3/users/016f7b40b0ce4c8297e65d943a65da4d => generated 0 bytes in 577 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:23:36.305332 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420a17e9-c839-4d8d-8435-54b57fb5bcf8 None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:36.305332 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420a17e9-c839-4d8d-8435-54b57fb5bcf8 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:36.305332 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420a17e9-c839-4d8d-8435-54b57fb5bcf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:36.323745 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-420a17e9-c839-4d8d-8435-54b57fb5bcf8 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:23:36.325113 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-420a17e9-c839-4d8d-8435-54b57fb5bcf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:23:36.386656 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 997/2005] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:23:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:23:36.397315 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d352b6-28f8-479c-95b3-135fe4eb7665 admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:36.399014 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-27d352b6-28f8-479c-95b3-135fe4eb7665 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:23:36.406574 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:23:36.468772 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d352b6-28f8-479c-95b3-135fe4eb7665 admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:36.472413 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca30c81-0268-4a2e-b773-a420810b69ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0817c3320d5d41d8bba0550c8ae8dd60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:36.473079 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca30c81-0268-4a2e-b773-a420810b69ec service neutron] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:36.473884 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca30c81-0268-4a2e-b773-a420810b69ec service neutron] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:36.474166 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca30c81-0268-4a2e-b773-a420810b69ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:36.483513 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca30c81-0268-4a2e-b773-a420810b69ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:36.484980 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca30c81-0268-4a2e-b773-a420810b69ec service neutron] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:36.494404 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1009/2006] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:23:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 01:23:37.034604 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420a17e9-c839-4d8d-8435-54b57fb5bcf8 None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:37.039612 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-420a17e9-c839-4d8d-8435-54b57fb5bcf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:23:37.054227 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420a17e9-c839-4d8d-8435-54b57fb5bcf8 None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:37.066770 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c26f01-bf62-4961-923d-bb968d16e530 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:37.068427 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c26f01-bf62-4961-923d-bb968d16e530 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:37.068908 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c26f01-bf62-4961-923d-bb968d16e530 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:37.069335 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c26f01-bf62-4961-923d-bb968d16e530 admin admin] PATH_INFO: `/v3/projects/7e7373ed73c540eb80de520de253800a` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:37.081007 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c26f01-bf62-4961-923d-bb968d16e530 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e7373ed73c540eb80de520de253800a)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:37.081007 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c26f01-bf62-4961-923d-bb968d16e530 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:37.149843 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:23:37.197144 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-66c26f01-bf62-4961-923d-bb968d16e530 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e7373ed73c540eb80de520de253800a', 'request_id': 'req-66c26f01-bf62-4961-923d-bb968d16e530'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:37.203401 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-66c26f01-bf62-4961-923d-bb968d16e530 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e7373ed73c540eb80de520de253800a', 'request_id': 'req-66c26f01-bf62-4961-923d-bb968d16e530'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:37.210747 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-66c26f01-bf62-4961-923d-bb968d16e530 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e7373ed73c540eb80de520de253800a', 'request_id': 'req-66c26f01-bf62-4961-923d-bb968d16e530'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:37.276758 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-66c26f01-bf62-4961-923d-bb968d16e530 admin admin] The token cache is being invalidate because project 7e7373ed73c540eb80de520de253800a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81749) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 01:23:37.277153 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-66c26f01-bf62-4961-923d-bb968d16e530 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:37.277815 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-66c26f01-bf62-4961-923d-bb968d16e530 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:37.279552 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 998/2007] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 01:23:37 2022] DELETE /identity/v3/projects/7e7373ed73c540eb80de520de253800a => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:23:37.283459 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca30c81-0268-4a2e-b773-a420810b69ec service neutron] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:37.293523 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca30c81-0268-4a2e-b773-a420810b69ec service neutron] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:37.352782 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a392c19d-0fe3-4ce9-b03d-680e0bfa76e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:37.353395 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a392c19d-0fe3-4ce9-b03d-680e0bfa76e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:37.353771 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a392c19d-0fe3-4ce9-b03d-680e0bfa76e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:37.354047 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a392c19d-0fe3-4ce9-b03d-680e0bfa76e4 admin admin] PATH_INFO: `/v3/users/1709a1e94c0e4979977cd6d30d418a69` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:37.355171 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a392c19d-0fe3-4ce9-b03d-680e0bfa76e4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1709a1e94c0e4979977cd6d30d418a69)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:37.357253 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a392c19d-0fe3-4ce9-b03d-680e0bfa76e4 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:37.608431 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-a392c19d-0fe3-4ce9-b03d-680e0bfa76e4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1709a1e94c0e4979977cd6d30d418a69'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:37.632534 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-a392c19d-0fe3-4ce9-b03d-680e0bfa76e4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1709a1e94c0e4979977cd6d30d418a69'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:37.633663 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-a392c19d-0fe3-4ce9-b03d-680e0bfa76e4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1709a1e94c0e4979977cd6d30d418a69'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:37.642156 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-a392c19d-0fe3-4ce9-b03d-680e0bfa76e4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1709a1e94c0e4979977cd6d30d418a69'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:37.728968 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2879e8e0-39c8-428b-9a5e-651abf04779c None None] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:37.729422 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2879e8e0-39c8-428b-9a5e-651abf04779c None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:37.729778 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2879e8e0-39c8-428b-9a5e-651abf04779c None None] PATH_INFO: `/` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:37.732786 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:23:37.736709 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 999/2008] 10.222.0.20 () {54 vars in 921 bytes} [Wed Aug 31 01:23:37 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 01:23:37.742470 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-a392c19d-0fe3-4ce9-b03d-680e0bfa76e4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1709a1e94c0e4979977cd6d30d418a69'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:37.749123 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1010/2009] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 01:23:37 2022] DELETE /identity/v3/users/1709a1e94c0e4979977cd6d30d418a69 => generated 0 bytes in 466 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:23:37.777502 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cfe878-ed14-470d-ae14-827637413e3c None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:37.777793 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cfe878-ed14-470d-ae14-827637413e3c None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:37.778076 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cfe878-ed14-470d-ae14-827637413e3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:37.807230 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-11cfe878-ed14-470d-ae14-827637413e3c None None] MFA Rules not processed for user `dea0fe5b1d6b405cb47b95192580303b`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:23:37.808476 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-11cfe878-ed14-470d-ae14-827637413e3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:23:37.874192 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1000/2010] 10.222.0.20 () {58 vars in 1018 bytes} [Wed Aug 31 01:23:37 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:23:37.885375 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a392c19d-0fe3-4ce9-b03d-680e0bfa76e4 admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:37.886824 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-a392c19d-0fe3-4ce9-b03d-680e0bfa76e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:23:37.903058 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a392c19d-0fe3-4ce9-b03d-680e0bfa76e4 admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:37.908398 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13162f8e-1453-4777-961d-c7f01c25bd53 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dea0fe5b1d6b405cb47b95192580303b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:37.909131 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13162f8e-1453-4777-961d-c7f01c25bd53 service placement] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:37.909459 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13162f8e-1453-4777-961d-c7f01c25bd53 service placement] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:37.909737 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13162f8e-1453-4777-961d-c7f01c25bd53 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:37.921398 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cfe878-ed14-470d-ae14-827637413e3c None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:37.922746 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-11cfe878-ed14-470d-ae14-827637413e3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:23:37.925619 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13162f8e-1453-4777-961d-c7f01c25bd53 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:37.927278 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13162f8e-1453-4777-961d-c7f01c25bd53 service placement] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:37.937594 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1011/2011] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:23:37 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 01:23:37.990814 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cfe878-ed14-470d-ae14-827637413e3c None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:37.995540 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba884412-22e9-41f9-981a-c60c6acd93a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:37.996169 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba884412-22e9-41f9-981a-c60c6acd93a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:37.996456 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba884412-22e9-41f9-981a-c60c6acd93a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:37.996789 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba884412-22e9-41f9-981a-c60c6acd93a1 admin admin] PATH_INFO: `/v3/users/b7d058c2b40343a6a3e2e46f6b2bab0e` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:37.998532 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba884412-22e9-41f9-981a-c60c6acd93a1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b7d058c2b40343a6a3e2e46f6b2bab0e)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:38.000126 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba884412-22e9-41f9-981a-c60c6acd93a1 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:38.151912 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=89,keystone:INSERT=5,keystone:DELETE=6 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:23:38.221922 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-ba884412-22e9-41f9-981a-c60c6acd93a1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b7d058c2b40343a6a3e2e46f6b2bab0e'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:38.259838 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13162f8e-1453-4777-961d-c7f01c25bd53 service placement] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:38.265889 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-13162f8e-1453-4777-961d-c7f01c25bd53 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:23:38.292583 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13162f8e-1453-4777-961d-c7f01c25bd53 service placement] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:38.299079 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886cd1e8-7201-4e50-a469-87e7be9ab55b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:38.299739 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886cd1e8-7201-4e50-a469-87e7be9ab55b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:38.300012 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886cd1e8-7201-4e50-a469-87e7be9ab55b admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:38.300478 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886cd1e8-7201-4e50-a469-87e7be9ab55b admin admin] PATH_INFO: `/v3/projects/321e86b471354116b0847bb7c8625664` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:38.301868 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886cd1e8-7201-4e50-a469-87e7be9ab55b admin admin] RBAC: Authorizing `identity:delete_project(project_id=321e86b471354116b0847bb7c8625664)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:38.304173 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886cd1e8-7201-4e50-a469-87e7be9ab55b admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:38.310129 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-ba884412-22e9-41f9-981a-c60c6acd93a1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b7d058c2b40343a6a3e2e46f6b2bab0e'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:38.311190 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-ba884412-22e9-41f9-981a-c60c6acd93a1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b7d058c2b40343a6a3e2e46f6b2bab0e'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:38.330038 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-ba884412-22e9-41f9-981a-c60c6acd93a1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b7d058c2b40343a6a3e2e46f6b2bab0e'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:38.331190 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-ba884412-22e9-41f9-981a-c60c6acd93a1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b7d058c2b40343a6a3e2e46f6b2bab0e'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:38.354321 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1001/2012] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 01:23:37 2022] DELETE /identity/v3/users/b7d058c2b40343a6a3e2e46f6b2bab0e => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 01:23:38.450286 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-886cd1e8-7201-4e50-a469-87e7be9ab55b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '321e86b471354116b0847bb7c8625664', 'request_id': 'req-886cd1e8-7201-4e50-a469-87e7be9ab55b'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:38.457164 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-886cd1e8-7201-4e50-a469-87e7be9ab55b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '321e86b471354116b0847bb7c8625664', 'request_id': 'req-886cd1e8-7201-4e50-a469-87e7be9ab55b'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:38.555678 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-886cd1e8-7201-4e50-a469-87e7be9ab55b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '321e86b471354116b0847bb7c8625664', 'request_id': 'req-886cd1e8-7201-4e50-a469-87e7be9ab55b'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:38.568207 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-886cd1e8-7201-4e50-a469-87e7be9ab55b admin admin] The token cache is being invalidate because project 321e86b471354116b0847bb7c8625664 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81750) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 01:23:38.568500 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-886cd1e8-7201-4e50-a469-87e7be9ab55b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:38.569184 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-886cd1e8-7201-4e50-a469-87e7be9ab55b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:38.570794 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1012/2013] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 01:23:38 2022] DELETE /identity/v3/projects/321e86b471354116b0847bb7c8625664 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:23:38.576871 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fa53c1-bd51-4657-8907-b34bad9c3f33 None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:38.577464 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fa53c1-bd51-4657-8907-b34bad9c3f33 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:38.577464 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fa53c1-bd51-4657-8907-b34bad9c3f33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:38.597504 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-b6fa53c1-bd51-4657-8907-b34bad9c3f33 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:23:38.602965 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-b6fa53c1-bd51-4657-8907-b34bad9c3f33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:23:38.627322 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886cd1e8-7201-4e50-a469-87e7be9ab55b admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:38.628943 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-886cd1e8-7201-4e50-a469-87e7be9ab55b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:23:38.668588 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1002/2014] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:23:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:23:38.676940 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc222413-c3fe-4a77-8a45-2e4a8cdc1f7a None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:38.677268 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc222413-c3fe-4a77-8a45-2e4a8cdc1f7a None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:38.677513 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc222413-c3fe-4a77-8a45-2e4a8cdc1f7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:38.699579 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-bc222413-c3fe-4a77-8a45-2e4a8cdc1f7a None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:23:38.700373 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886cd1e8-7201-4e50-a469-87e7be9ab55b admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:38.700826 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-bc222413-c3fe-4a77-8a45-2e4a8cdc1f7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:23:38.706289 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b85b5c-8c14-4cd6-b0dd-4f97de8a4fac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:38.712312 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b85b5c-8c14-4cd6-b0dd-4f97de8a4fac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:38.712312 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b85b5c-8c14-4cd6-b0dd-4f97de8a4fac admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:38.712312 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b85b5c-8c14-4cd6-b0dd-4f97de8a4fac admin admin] PATH_INFO: `/v3/projects/3abb95e8d0a84b76a5d141815451ceae` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:38.712312 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b85b5c-8c14-4cd6-b0dd-4f97de8a4fac admin admin] RBAC: Authorizing `identity:delete_project(project_id=3abb95e8d0a84b76a5d141815451ceae)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:38.712312 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b85b5c-8c14-4cd6-b0dd-4f97de8a4fac admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:38.713943 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1003/2015] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:23:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 01:23:38.715044 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc222413-c3fe-4a77-8a45-2e4a8cdc1f7a None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:38.725588 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc222413-c3fe-4a77-8a45-2e4a8cdc1f7a None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:38.744019 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a69a7e-dd9b-4258-869f-31469731989a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:38.744910 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a69a7e-dd9b-4258-869f-31469731989a admin admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:38.745124 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a69a7e-dd9b-4258-869f-31469731989a admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:38.745409 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a69a7e-dd9b-4258-869f-31469731989a admin admin] PATH_INFO: `/v3/domains` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:38.746068 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a69a7e-dd9b-4258-869f-31469731989a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:23:38.746379 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a69a7e-dd9b-4258-869f-31469731989a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:38.748041 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a69a7e-dd9b-4258-869f-31469731989a admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:38.764633 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1004/2016] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Aug 31 01:23:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:23:38.772512 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a69a7e-dd9b-4258-869f-31469731989a admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:38.783159 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a69a7e-dd9b-4258-869f-31469731989a admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:38.788667 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5a6a8a-0a05-495e-8504-b83ef98467dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:38.789303 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5a6a8a-0a05-495e-8504-b83ef98467dc admin admin] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:38.789588 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5a6a8a-0a05-495e-8504-b83ef98467dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:38.789863 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5a6a8a-0a05-495e-8504-b83ef98467dc admin admin] PATH_INFO: `/v3/projects` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:38.790644 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5a6a8a-0a05-495e-8504-b83ef98467dc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:38.792897 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5a6a8a-0a05-495e-8504-b83ef98467dc admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:38.842752 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-05b85b5c-8c14-4cd6-b0dd-4f97de8a4fac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3abb95e8d0a84b76a5d141815451ceae', 'request_id': 'req-05b85b5c-8c14-4cd6-b0dd-4f97de8a4fac'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:38.848049 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-05b85b5c-8c14-4cd6-b0dd-4f97de8a4fac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3abb95e8d0a84b76a5d141815451ceae', 'request_id': 'req-05b85b5c-8c14-4cd6-b0dd-4f97de8a4fac'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:38.907293 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1005/2017] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 01:23:38 2022] POST /identity/v3/projects => generated 369 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:23:38.908145 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5a6a8a-0a05-495e-8504-b83ef98467dc admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:38.909778 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-4b5a6a8a-0a05-495e-8504-b83ef98467dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:23:38.932321 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-05b85b5c-8c14-4cd6-b0dd-4f97de8a4fac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3abb95e8d0a84b76a5d141815451ceae', 'request_id': 'req-05b85b5c-8c14-4cd6-b0dd-4f97de8a4fac'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:38.945508 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-05b85b5c-8c14-4cd6-b0dd-4f97de8a4fac admin admin] The token cache is being invalidate because project 3abb95e8d0a84b76a5d141815451ceae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81750) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 01:23:38.947522 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-05b85b5c-8c14-4cd6-b0dd-4f97de8a4fac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:38.947522 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-05b85b5c-8c14-4cd6-b0dd-4f97de8a4fac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:38.948210 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1013/2018] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 01:23:38 2022] DELETE /identity/v3/projects/3abb95e8d0a84b76a5d141815451ceae => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 01:23:38.949004 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b85b5c-8c14-4cd6-b0dd-4f97de8a4fac admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:38.950318 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-05b85b5c-8c14-4cd6-b0dd-4f97de8a4fac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:23:38.992533 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5a6a8a-0a05-495e-8504-b83ef98467dc admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:38.992678 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8292266-0c93-45d7-9233-b164a27f413a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5c8fc69c518412ba1aad61a9f4f558c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:38.993359 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8292266-0c93-45d7-9233-b164a27f413a service cinder] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:38.993834 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8292266-0c93-45d7-9233-b164a27f413a service cinder] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:38.994555 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8292266-0c93-45d7-9233-b164a27f413a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:38.996068 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-b8292266-0c93-45d7-9233-b164a27f413a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:23:39.033465 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b85b5c-8c14-4cd6-b0dd-4f97de8a4fac admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:39.068258 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8292266-0c93-45d7-9233-b164a27f413a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:39.070064 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8292266-0c93-45d7-9233-b164a27f413a service cinder] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:39.080206 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1006/2019] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:23:38 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:23:39.081583 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbaa9604-a280-412c-9d2a-7767cc80804d None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:39.081875 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbaa9604-a280-412c-9d2a-7767cc80804d None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:39.082154 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbaa9604-a280-412c-9d2a-7767cc80804d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:39.091709 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33499166-ee8d-4174-8018-1953df02819a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:39.092333 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33499166-ee8d-4174-8018-1953df02819a admin admin] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:39.092611 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33499166-ee8d-4174-8018-1953df02819a admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:39.092932 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33499166-ee8d-4174-8018-1953df02819a admin admin] PATH_INFO: `/v3/users` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:39.093566 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33499166-ee8d-4174-8018-1953df02819a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:39.095536 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33499166-ee8d-4174-8018-1953df02819a admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:39.102915 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-bbaa9604-a280-412c-9d2a-7767cc80804d None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:23:39.104358 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-bbaa9604-a280-412c-9d2a-7767cc80804d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:23:39.112575 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1007/2020] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:23:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:23:39.120569 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f39305-b1ef-4ae8-bced-b1220dc9e70a None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:39.120923 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f39305-b1ef-4ae8-bced-b1220dc9e70a None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:39.126357 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f39305-b1ef-4ae8-bced-b1220dc9e70a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:39.140782 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-a9f39305-b1ef-4ae8-bced-b1220dc9e70a None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:23:39.142658 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-a9f39305-b1ef-4ae8-bced-b1220dc9e70a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:23:39.151577 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1008/2021] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:23:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 01:23:39.155756 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f39305-b1ef-4ae8-bced-b1220dc9e70a None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:39.165144 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f39305-b1ef-4ae8-bced-b1220dc9e70a None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:39.169883 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949673bf-9ba7-4ddf-8008-e25f0c51300d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:39.170499 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949673bf-9ba7-4ddf-8008-e25f0c51300d admin admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:39.170777 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949673bf-9ba7-4ddf-8008-e25f0c51300d admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:39.171058 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949673bf-9ba7-4ddf-8008-e25f0c51300d admin admin] PATH_INFO: `/v3/domains` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:39.171692 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949673bf-9ba7-4ddf-8008-e25f0c51300d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:23:39.171994 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949673bf-9ba7-4ddf-8008-e25f0c51300d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:39.173703 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949673bf-9ba7-4ddf-8008-e25f0c51300d admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:39.185875 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1009/2022] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Aug 31 01:23:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 01:23:39.190252 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949673bf-9ba7-4ddf-8008-e25f0c51300d admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:39.202871 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949673bf-9ba7-4ddf-8008-e25f0c51300d admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:39.205516 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1014/2023] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 01:23:38 2022] POST /identity/v3/users => generated 285 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:23:39.210189 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33499166-ee8d-4174-8018-1953df02819a admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:39.211875 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613b5051-8597-4965-bd5a-1e52625f5839 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:39.212524 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613b5051-8597-4965-bd5a-1e52625f5839 admin admin] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:39.212843 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613b5051-8597-4965-bd5a-1e52625f5839 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:39.213123 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613b5051-8597-4965-bd5a-1e52625f5839 admin admin] PATH_INFO: `/v3/projects` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:39.213757 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613b5051-8597-4965-bd5a-1e52625f5839 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:39.215843 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613b5051-8597-4965-bd5a-1e52625f5839 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:39.220483 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33499166-ee8d-4174-8018-1953df02819a admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:39.225213 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dd7919-e5bb-444c-a3d9-60fd27ea6002 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:39.225816 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dd7919-e5bb-444c-a3d9-60fd27ea6002 admin admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:39.226100 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dd7919-e5bb-444c-a3d9-60fd27ea6002 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:39.226380 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dd7919-e5bb-444c-a3d9-60fd27ea6002 admin admin] PATH_INFO: `/v3/roles` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:39.227020 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0dd7919-e5bb-444c-a3d9-60fd27ea6002 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:39.229003 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0dd7919-e5bb-444c-a3d9-60fd27ea6002 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:39.240757 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1015/2024] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:23:39 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 01:23:39.243958 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dd7919-e5bb-444c-a3d9-60fd27ea6002 admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:39.253626 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dd7919-e5bb-444c-a3d9-60fd27ea6002 admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:39.259134 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f42f943-ca14-4221-a41b-28bd2b1a28ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:39.259728 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f42f943-ca14-4221-a41b-28bd2b1a28ce admin admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:39.260008 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f42f943-ca14-4221-a41b-28bd2b1a28ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:39.260282 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f42f943-ca14-4221-a41b-28bd2b1a28ce admin admin] PATH_INFO: `/v3/roles` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:39.260948 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f42f943-ca14-4221-a41b-28bd2b1a28ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:39.262543 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f42f943-ca14-4221-a41b-28bd2b1a28ce admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:39.273186 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1016/2025] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:23:39 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 01:23:39.276995 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f42f943-ca14-4221-a41b-28bd2b1a28ce admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:39.286948 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f42f943-ca14-4221-a41b-28bd2b1a28ce admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:39.291957 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062cb09d-09b7-4af7-abcd-9edd04e85127 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:39.292698 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062cb09d-09b7-4af7-abcd-9edd04e85127 admin admin] REQUEST_METHOD: `PUT` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:39.293085 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062cb09d-09b7-4af7-abcd-9edd04e85127 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:39.293366 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062cb09d-09b7-4af7-abcd-9edd04e85127 admin admin] PATH_INFO: `/v3/projects/bea70716ec01430184addb63a7558a9d/users/702fa5375a84438d8ce942198866873d/roles/d0b9768f4d014766b9326387f2e11f34` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:39.310310 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062cb09d-09b7-4af7-abcd-9edd04e85127 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bea70716ec01430184addb63a7558a9d, user_id=702fa5375a84438d8ce942198866873d, role_id=d0b9768f4d014766b9326387f2e11f34)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:39.313200 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062cb09d-09b7-4af7-abcd-9edd04e85127 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:39.322267 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1010/2026] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 01:23:39 2022] POST /identity/v3/projects => generated 377 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:23:39.323338 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613b5051-8597-4965-bd5a-1e52625f5839 admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:39.404323 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613b5051-8597-4965-bd5a-1e52625f5839 admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:39.412676 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e03a321-eb8d-40d9-991a-625dc5f7b03c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:39.413429 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e03a321-eb8d-40d9-991a-625dc5f7b03c admin admin] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:39.413812 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e03a321-eb8d-40d9-991a-625dc5f7b03c admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:39.414191 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e03a321-eb8d-40d9-991a-625dc5f7b03c admin admin] PATH_INFO: `/v3/users` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:39.414920 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e03a321-eb8d-40d9-991a-625dc5f7b03c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:39.417503 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e03a321-eb8d-40d9-991a-625dc5f7b03c admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:39.423891 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1017/2027] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 01:23:39 2022] PUT /identity/v3/projects/bea70716ec01430184addb63a7558a9d/users/702fa5375a84438d8ce942198866873d/roles/d0b9768f4d014766b9326387f2e11f34 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 01:23:39.436683 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f35572-b158-4f09-9155-e3ff7175a78f None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:39.437008 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f35572-b158-4f09-9155-e3ff7175a78f None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:39.437289 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f35572-b158-4f09-9155-e3ff7175a78f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:39.466235 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-38f35572-b158-4f09-9155-e3ff7175a78f None None] MFA Rules not processed for user `702fa5375a84438d8ce942198866873d`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:23:39.467409 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-38f35572-b158-4f09-9155-e3ff7175a78f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:23:39.523810 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1011/2028] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 01:23:39 2022] POST /identity/v3/users => generated 289 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:23:39.524928 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e03a321-eb8d-40d9-991a-625dc5f7b03c admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:39.528964 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1018/2029] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:23:39 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 01:23:39.533189 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f35572-b158-4f09-9155-e3ff7175a78f None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:39.534729 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e03a321-eb8d-40d9-991a-625dc5f7b03c admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:39.539043 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:23:39.604170 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7a8dc5-6a70-48a0-bce5-5d494b924f22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:39.616904 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7a8dc5-6a70-48a0-bce5-5d494b924f22 admin admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:39.616904 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7a8dc5-6a70-48a0-bce5-5d494b924f22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:39.616904 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7a8dc5-6a70-48a0-bce5-5d494b924f22 admin admin] PATH_INFO: `/v3/roles` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:39.616904 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7a8dc5-6a70-48a0-bce5-5d494b924f22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:39.616904 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7a8dc5-6a70-48a0-bce5-5d494b924f22 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:39.617353 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f35572-b158-4f09-9155-e3ff7175a78f None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:39.617353 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d8b910-aeec-40f4-9ad8-b193f17d1e3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:39.617353 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d8b910-aeec-40f4-9ad8-b193f17d1e3e admin admin] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:39.617353 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d8b910-aeec-40f4-9ad8-b193f17d1e3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:39.617687 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d8b910-aeec-40f4-9ad8-b193f17d1e3e admin admin] PATH_INFO: `/v3/projects` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:39.619652 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d8b910-aeec-40f4-9ad8-b193f17d1e3e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:39.620315 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d8b910-aeec-40f4-9ad8-b193f17d1e3e admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:39.620823 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1012/2030] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:23:39 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:23:39.624379 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7a8dc5-6a70-48a0-bce5-5d494b924f22 admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:39.636358 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7a8dc5-6a70-48a0-bce5-5d494b924f22 admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:39.642078 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b80fe3-220a-4a83-abb9-13fa325aaa25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:39.642709 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b80fe3-220a-4a83-abb9-13fa325aaa25 admin admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:39.642991 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b80fe3-220a-4a83-abb9-13fa325aaa25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:39.643266 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b80fe3-220a-4a83-abb9-13fa325aaa25 admin admin] PATH_INFO: `/v3/roles` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:39.644016 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b80fe3-220a-4a83-abb9-13fa325aaa25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:39.645557 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b80fe3-220a-4a83-abb9-13fa325aaa25 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:39.657227 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1013/2031] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:23:39 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 01:23:39.661665 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b80fe3-220a-4a83-abb9-13fa325aaa25 admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:39.671490 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b80fe3-220a-4a83-abb9-13fa325aaa25 admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:39.677162 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b101fd-cee7-4a4d-a809-6892a70f8ac2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:39.677810 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b101fd-cee7-4a4d-a809-6892a70f8ac2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:39.678090 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b101fd-cee7-4a4d-a809-6892a70f8ac2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:39.678365 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b101fd-cee7-4a4d-a809-6892a70f8ac2 admin admin] PATH_INFO: `/v3/projects/fb863f706c3846db958c9b7fe367c3f5/users/1cc3db55df284cd39f28fcfefc619fda/roles/d0b9768f4d014766b9326387f2e11f34` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:39.692510 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b101fd-cee7-4a4d-a809-6892a70f8ac2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb863f706c3846db958c9b7fe367c3f5, user_id=1cc3db55df284cd39f28fcfefc619fda, role_id=d0b9768f4d014766b9326387f2e11f34)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:39.695214 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b101fd-cee7-4a4d-a809-6892a70f8ac2 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:39.793385 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1019/2032] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 01:23:39 2022] POST /identity/v3/projects => generated 365 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 01:23:39.797288 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d8b910-aeec-40f4-9ad8-b193f17d1e3e admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:39.812945 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:23:39.825166 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1014/2033] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 01:23:39 2022] PUT /identity/v3/projects/fb863f706c3846db958c9b7fe367c3f5/users/1cc3db55df284cd39f28fcfefc619fda/roles/d0b9768f4d014766b9326387f2e11f34 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 01:23:39.830605 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08269e17-f7e8-4391-8a87-92ddea47c513 None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:39.831037 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08269e17-f7e8-4391-8a87-92ddea47c513 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:39.831319 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08269e17-f7e8-4391-8a87-92ddea47c513 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:39.852147 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-08269e17-f7e8-4391-8a87-92ddea47c513 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:23:39.855355 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-08269e17-f7e8-4391-8a87-92ddea47c513 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:23:39.871382 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d8b910-aeec-40f4-9ad8-b193f17d1e3e admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:39.935822 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc81f3c-020f-4bcf-b9f8-99cc296b5edc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:39.936567 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc81f3c-020f-4bcf-b9f8-99cc296b5edc admin admin] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:39.936916 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc81f3c-020f-4bcf-b9f8-99cc296b5edc admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:39.937398 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc81f3c-020f-4bcf-b9f8-99cc296b5edc admin admin] PATH_INFO: `/v3/users` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:39.937831 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc81f3c-020f-4bcf-b9f8-99cc296b5edc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:39.939739 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc81f3c-020f-4bcf-b9f8-99cc296b5edc admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:39.957559 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1015/2034] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:23:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 01:23:39.967126 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08269e17-f7e8-4391-8a87-92ddea47c513 None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:39.968620 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-08269e17-f7e8-4391-8a87-92ddea47c513 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:23:40.030101 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08269e17-f7e8-4391-8a87-92ddea47c513 None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:40.033841 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a1cc4b-0d23-4aee-a8ae-67ac49e14be1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0817c3320d5d41d8bba0550c8ae8dd60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:40.034453 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a1cc4b-0d23-4aee-a8ae-67ac49e14be1 service neutron] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:40.034742 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a1cc4b-0d23-4aee-a8ae-67ac49e14be1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:40.035022 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a1cc4b-0d23-4aee-a8ae-67ac49e14be1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:40.045128 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a1cc4b-0d23-4aee-a8ae-67ac49e14be1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:40.046508 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a1cc4b-0d23-4aee-a8ae-67ac49e14be1 service neutron] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:40.057401 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1016/2035] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:23:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 01:23:40.088517 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1020/2036] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 01:23:39 2022] POST /identity/v3/users => generated 289 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 01:23:40.092462 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a1cc4b-0d23-4aee-a8ae-67ac49e14be1 service neutron] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:40.102581 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a1cc4b-0d23-4aee-a8ae-67ac49e14be1 service neutron] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:40.107357 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8f09ac-dbae-4a9d-932f-e5ffe44f4f25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:40.107960 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8f09ac-dbae-4a9d-932f-e5ffe44f4f25 admin admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:40.108244 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8f09ac-dbae-4a9d-932f-e5ffe44f4f25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:40.108516 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8f09ac-dbae-4a9d-932f-e5ffe44f4f25 admin admin] PATH_INFO: `/v3/roles` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:40.109207 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8f09ac-dbae-4a9d-932f-e5ffe44f4f25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:40.111224 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8f09ac-dbae-4a9d-932f-e5ffe44f4f25 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:40.124778 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1017/2037] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:23:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:23:40.126190 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc81f3c-020f-4bcf-b9f8-99cc296b5edc admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:40.137002 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc81f3c-020f-4bcf-b9f8-99cc296b5edc admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:40.145785 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d73665a-2155-4e2b-9dc2-c8b3821a80db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:40.145785 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d73665a-2155-4e2b-9dc2-c8b3821a80db admin admin] REQUEST_METHOD: `PUT` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:40.145785 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d73665a-2155-4e2b-9dc2-c8b3821a80db admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:40.145785 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d73665a-2155-4e2b-9dc2-c8b3821a80db admin admin] PATH_INFO: `/v3/projects/b34f5a84821d433d94c7569873161da6/users/784b68db20564825b663bccb5782aa34/roles/d009411496b8452b88e77ffb8db015a7` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:40.158592 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d73665a-2155-4e2b-9dc2-c8b3821a80db admin admin] RBAC: Authorizing `identity:create_grant(project_id=b34f5a84821d433d94c7569873161da6, user_id=784b68db20564825b663bccb5782aa34, role_id=d009411496b8452b88e77ffb8db015a7)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:40.162107 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d73665a-2155-4e2b-9dc2-c8b3821a80db admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:40.311683 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1021/2038] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 01:23:40 2022] PUT /identity/v3/projects/b34f5a84821d433d94c7569873161da6/users/784b68db20564825b663bccb5782aa34/roles/d009411496b8452b88e77ffb8db015a7 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:23:40.315570 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8f09ac-dbae-4a9d-932f-e5ffe44f4f25 admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:40.325518 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8f09ac-dbae-4a9d-932f-e5ffe44f4f25 admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:40.391592 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f521578b-feef-48b2-8c25-6ffa4a4a7a08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:40.392237 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f521578b-feef-48b2-8c25-6ffa4a4a7a08 admin admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:40.392520 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f521578b-feef-48b2-8c25-6ffa4a4a7a08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:40.392830 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f521578b-feef-48b2-8c25-6ffa4a4a7a08 admin admin] PATH_INFO: `/v3/roles` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:40.393464 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f521578b-feef-48b2-8c25-6ffa4a4a7a08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:40.395886 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f521578b-feef-48b2-8c25-6ffa4a4a7a08 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:40.406352 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1018/2039] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:23:40 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:23:40.410660 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d73665a-2155-4e2b-9dc2-c8b3821a80db admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:40.421612 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d73665a-2155-4e2b-9dc2-c8b3821a80db admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:40.427375 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce13ae2-63f6-4b73-af0d-bd70548ca3a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:40.428389 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce13ae2-63f6-4b73-af0d-bd70548ca3a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:40.428803 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce13ae2-63f6-4b73-af0d-bd70548ca3a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:40.429276 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce13ae2-63f6-4b73-af0d-bd70548ca3a6 admin admin] PATH_INFO: `/v3/projects/b34f5a84821d433d94c7569873161da6/users/784b68db20564825b663bccb5782aa34/roles/d009411496b8452b88e77ffb8db015a7` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:40.433073 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce13ae2-63f6-4b73-af0d-bd70548ca3a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b34f5a84821d433d94c7569873161da6, user_id=784b68db20564825b663bccb5782aa34, role_id=d009411496b8452b88e77ffb8db015a7)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:40.436298 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce13ae2-63f6-4b73-af0d-bd70548ca3a6 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:40.451788 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1022/2040] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 01:23:40 2022] PUT /identity/v3/projects/b34f5a84821d433d94c7569873161da6/users/784b68db20564825b663bccb5782aa34/roles/d009411496b8452b88e77ffb8db015a7 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:23:40.463530 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e426b045-1474-44ad-a682-e19128029f72 None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:40.463858 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e426b045-1474-44ad-a682-e19128029f72 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:40.464757 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e426b045-1474-44ad-a682-e19128029f72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:40.493552 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-e426b045-1474-44ad-a682-e19128029f72 None None] MFA Rules not processed for user `784b68db20564825b663bccb5782aa34`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:23:40.494770 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-e426b045-1474-44ad-a682-e19128029f72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:23:40.565170 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1019/2041] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:23:40 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:23:40.577681 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce13ae2-63f6-4b73-af0d-bd70548ca3a6 admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:40.579893 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-bce13ae2-63f6-4b73-af0d-bd70548ca3a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:23:40.646811 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce13ae2-63f6-4b73-af0d-bd70548ca3a6 admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:40.651027 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0199ca63-114e-4fbc-9ce3-f0c4ca258609 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5c8fc69c518412ba1aad61a9f4f558c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:40.651940 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0199ca63-114e-4fbc-9ce3-f0c4ca258609 service cinder] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:40.652398 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0199ca63-114e-4fbc-9ce3-f0c4ca258609 service cinder] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:40.652882 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0199ca63-114e-4fbc-9ce3-f0c4ca258609 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:40.668565 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0199ca63-114e-4fbc-9ce3-f0c4ca258609 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:40.670367 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0199ca63-114e-4fbc-9ce3-f0c4ca258609 service cinder] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:40.681862 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1023/2042] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:23:40 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:23:41.014299 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:23:41.680748 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:23:42.620484 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:DELETE=16,keystone:SELECT=82,keystone:INSERT=2 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:23:44.052707 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:23:44.996752 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:23:45.298081 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:23:45.619101 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e426b045-1474-44ad-a682-e19128029f72 None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:45.721997 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e426b045-1474-44ad-a682-e19128029f72 None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:45.731176 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8485f98c-b924-4135-bed6-0b2e99297252 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:45.731176 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8485f98c-b924-4135-bed6-0b2e99297252 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:45.731176 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8485f98c-b924-4135-bed6-0b2e99297252 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:45.731176 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8485f98c-b924-4135-bed6-0b2e99297252 admin admin] PATH_INFO: `/v3/users/702fa5375a84438d8ce942198866873d` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:45.731574 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8485f98c-b924-4135-bed6-0b2e99297252 admin admin] RBAC: Authorizing `identity:delete_user(user_id=702fa5375a84438d8ce942198866873d)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:45.732638 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8485f98c-b924-4135-bed6-0b2e99297252 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:46.001993 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-8485f98c-b924-4135-bed6-0b2e99297252 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '702fa5375a84438d8ce942198866873d'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:46.088661 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-8485f98c-b924-4135-bed6-0b2e99297252 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '702fa5375a84438d8ce942198866873d'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:46.090959 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-8485f98c-b924-4135-bed6-0b2e99297252 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '702fa5375a84438d8ce942198866873d'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:46.107693 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-8485f98c-b924-4135-bed6-0b2e99297252 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '702fa5375a84438d8ce942198866873d'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:46.108478 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-8485f98c-b924-4135-bed6-0b2e99297252 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '702fa5375a84438d8ce942198866873d'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:46.158267 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1020/2043] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 01:23:45 2022] DELETE /identity/v3/users/702fa5375a84438d8ce942198866873d => generated 0 bytes in 539 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:23:46.162714 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5c94e8-698d-443e-beed-7bfd88aa8f6d None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:46.164101 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5c94e8-698d-443e-beed-7bfd88aa8f6d None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:46.164579 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5c94e8-698d-443e-beed-7bfd88aa8f6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:46.201863 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-7c5c94e8-698d-443e-beed-7bfd88aa8f6d None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:23:46.206369 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-7c5c94e8-698d-443e-beed-7bfd88aa8f6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:23:46.326796 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1024/2044] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:23:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:23:46.349276 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8485f98c-b924-4135-bed6-0b2e99297252 admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:46.350811 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-8485f98c-b924-4135-bed6-0b2e99297252 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:23:46.460901 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8485f98c-b924-4135-bed6-0b2e99297252 admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:46.464413 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c00f314-735d-4298-8240-bfd3d8eb5b80 None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:46.465353 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87171c14-e3ef-41f3-8d4a-ac09de14eb22 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0817c3320d5d41d8bba0550c8ae8dd60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:46.465792 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c00f314-735d-4298-8240-bfd3d8eb5b80 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:46.466249 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c00f314-735d-4298-8240-bfd3d8eb5b80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:46.466382 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87171c14-e3ef-41f3-8d4a-ac09de14eb22 service neutron] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:46.466537 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87171c14-e3ef-41f3-8d4a-ac09de14eb22 service neutron] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:46.466818 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87171c14-e3ef-41f3-8d4a-ac09de14eb22 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:46.484437 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87171c14-e3ef-41f3-8d4a-ac09de14eb22 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:46.486044 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87171c14-e3ef-41f3-8d4a-ac09de14eb22 service neutron] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:46.501992 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1021/2045] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:23:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:23:46.510928 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-3c00f314-735d-4298-8240-bfd3d8eb5b80 None None] MFA Rules not processed for user `1cc3db55df284cd39f28fcfefc619fda`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:23:46.513056 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-3c00f314-735d-4298-8240-bfd3d8eb5b80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:23:46.631407 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1025/2046] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:23:46 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:23:46.650968 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87171c14-e3ef-41f3-8d4a-ac09de14eb22 service neutron] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:46.650968 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-87171c14-e3ef-41f3-8d4a-ac09de14eb22 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:23:46.737619 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87171c14-e3ef-41f3-8d4a-ac09de14eb22 service neutron] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:46.752958 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac29809-8acb-4459-adc0-103d400148d0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8744c9de803048f583d0317d01f6a540', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:46.755407 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac29809-8acb-4459-adc0-103d400148d0 service nova] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:46.755858 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac29809-8acb-4459-adc0-103d400148d0 service nova] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:46.756518 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac29809-8acb-4459-adc0-103d400148d0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:46.777911 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac29809-8acb-4459-adc0-103d400148d0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:46.777911 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac29809-8acb-4459-adc0-103d400148d0 service nova] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:46.802563 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:23:46.812298 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1022/2047] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:23:46 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:23:47.118789 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c00f314-735d-4298-8240-bfd3d8eb5b80 None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:47.120333 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-3c00f314-735d-4298-8240-bfd3d8eb5b80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:23:47.138746 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c00f314-735d-4298-8240-bfd3d8eb5b80 None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:47.149815 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba700729-577b-4fc2-9cc4-0a5ae4411b89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:47.149815 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba700729-577b-4fc2-9cc4-0a5ae4411b89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:47.149815 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba700729-577b-4fc2-9cc4-0a5ae4411b89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:47.149815 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba700729-577b-4fc2-9cc4-0a5ae4411b89 admin admin] PATH_INFO: `/v3/projects/bea70716ec01430184addb63a7558a9d` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:47.150405 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba700729-577b-4fc2-9cc4-0a5ae4411b89 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bea70716ec01430184addb63a7558a9d)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:47.150405 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba700729-577b-4fc2-9cc4-0a5ae4411b89 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:47.280178 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:23:47.306461 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-ba700729-577b-4fc2-9cc4-0a5ae4411b89 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bea70716ec01430184addb63a7558a9d', 'request_id': 'req-ba700729-577b-4fc2-9cc4-0a5ae4411b89'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:47.313817 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-ba700729-577b-4fc2-9cc4-0a5ae4411b89 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bea70716ec01430184addb63a7558a9d', 'request_id': 'req-ba700729-577b-4fc2-9cc4-0a5ae4411b89'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:47.419647 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-ba700729-577b-4fc2-9cc4-0a5ae4411b89 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bea70716ec01430184addb63a7558a9d', 'request_id': 'req-ba700729-577b-4fc2-9cc4-0a5ae4411b89'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:47.434056 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-ba700729-577b-4fc2-9cc4-0a5ae4411b89 admin admin] The token cache is being invalidate because project bea70716ec01430184addb63a7558a9d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81750) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 01:23:47.434604 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-ba700729-577b-4fc2-9cc4-0a5ae4411b89 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:47.435405 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-ba700729-577b-4fc2-9cc4-0a5ae4411b89 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:47.437859 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1026/2048] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 01:23:47 2022] DELETE /identity/v3/projects/bea70716ec01430184addb63a7558a9d => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:23:47.443204 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac29809-8acb-4459-adc0-103d400148d0 service nova] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:47.454491 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac29809-8acb-4459-adc0-103d400148d0 service nova] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:47.545278 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e3c8a1-4d55-4ecd-8484-b84daa02af51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:47.546503 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e3c8a1-4d55-4ecd-8484-b84daa02af51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:47.546503 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e3c8a1-4d55-4ecd-8484-b84daa02af51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:47.546503 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e3c8a1-4d55-4ecd-8484-b84daa02af51 admin admin] PATH_INFO: `/v3/users/784b68db20564825b663bccb5782aa34` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:47.548501 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e3c8a1-4d55-4ecd-8484-b84daa02af51 admin admin] RBAC: Authorizing `identity:delete_user(user_id=784b68db20564825b663bccb5782aa34)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:47.552385 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e3c8a1-4d55-4ecd-8484-b84daa02af51 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:47.830962 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-26e3c8a1-4d55-4ecd-8484-b84daa02af51 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '784b68db20564825b663bccb5782aa34'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:47.899507 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-26e3c8a1-4d55-4ecd-8484-b84daa02af51 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '784b68db20564825b663bccb5782aa34'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:47.900443 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-26e3c8a1-4d55-4ecd-8484-b84daa02af51 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '784b68db20564825b663bccb5782aa34'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:47.914727 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-26e3c8a1-4d55-4ecd-8484-b84daa02af51 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '784b68db20564825b663bccb5782aa34'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:47.915608 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-26e3c8a1-4d55-4ecd-8484-b84daa02af51 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '784b68db20564825b663bccb5782aa34'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:47.941696 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1023/2049] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 01:23:47 2022] DELETE /identity/v3/users/784b68db20564825b663bccb5782aa34 => generated 0 bytes in 496 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:23:48.550996 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba700729-577b-4fc2-9cc4-0a5ae4411b89 admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:48.552457 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-ba700729-577b-4fc2-9cc4-0a5ae4411b89 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:23:48.580245 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba700729-577b-4fc2-9cc4-0a5ae4411b89 admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:48.605853 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e3c8a1-4d55-4ecd-8484-b84daa02af51 admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:48.608014 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-26e3c8a1-4d55-4ecd-8484-b84daa02af51 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:23:48.623918 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:23:48.682195 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c784d3cd-23ed-41c5-83e9-487beaef62ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:48.682195 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c784d3cd-23ed-41c5-83e9-487beaef62ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:48.682195 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c784d3cd-23ed-41c5-83e9-487beaef62ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:48.682195 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c784d3cd-23ed-41c5-83e9-487beaef62ec admin admin] PATH_INFO: `/v3/projects/b34f5a84821d433d94c7569873161da6` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:48.682676 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c784d3cd-23ed-41c5-83e9-487beaef62ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=b34f5a84821d433d94c7569873161da6)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:48.685156 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c784d3cd-23ed-41c5-83e9-487beaef62ec admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:48.732377 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e3c8a1-4d55-4ecd-8484-b84daa02af51 admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:48.738092 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b453d93-d26c-47e4-929a-85140618b1c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:48.739412 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b453d93-d26c-47e4-929a-85140618b1c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:48.739866 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b453d93-d26c-47e4-929a-85140618b1c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:48.740402 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b453d93-d26c-47e4-929a-85140618b1c9 admin admin] PATH_INFO: `/v3/users/91f71abcbc54454da9a52debf048ff0f` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:48.742125 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b453d93-d26c-47e4-929a-85140618b1c9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=91f71abcbc54454da9a52debf048ff0f)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:48.745034 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b453d93-d26c-47e4-929a-85140618b1c9 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:48.871483 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-c784d3cd-23ed-41c5-83e9-487beaef62ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b34f5a84821d433d94c7569873161da6', 'request_id': 'req-c784d3cd-23ed-41c5-83e9-487beaef62ec'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:48.913702 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-c784d3cd-23ed-41c5-83e9-487beaef62ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b34f5a84821d433d94c7569873161da6', 'request_id': 'req-c784d3cd-23ed-41c5-83e9-487beaef62ec'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:48.996908 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-c784d3cd-23ed-41c5-83e9-487beaef62ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b34f5a84821d433d94c7569873161da6', 'request_id': 'req-c784d3cd-23ed-41c5-83e9-487beaef62ec'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:49.017051 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-c784d3cd-23ed-41c5-83e9-487beaef62ec admin admin] The token cache is being invalidate because project b34f5a84821d433d94c7569873161da6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81750) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 01:23:49.017441 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-c784d3cd-23ed-41c5-83e9-487beaef62ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:49.018205 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-c784d3cd-23ed-41c5-83e9-487beaef62ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:49.020900 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1027/2050] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 01:23:48 2022] DELETE /identity/v3/projects/b34f5a84821d433d94c7569873161da6 => generated 0 bytes in 470 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:23:49.020900 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c784d3cd-23ed-41c5-83e9-487beaef62ec admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:49.020900 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-c784d3cd-23ed-41c5-83e9-487beaef62ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:23:49.031055 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-0b453d93-d26c-47e4-929a-85140618b1c9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '91f71abcbc54454da9a52debf048ff0f'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:49.100333 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c784d3cd-23ed-41c5-83e9-487beaef62ec admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:49.106232 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bc02ae-954c-4d84-9b67-5d13c1ef390e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:49.106989 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bc02ae-954c-4d84-9b67-5d13c1ef390e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:49.107367 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bc02ae-954c-4d84-9b67-5d13c1ef390e admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:49.107700 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bc02ae-954c-4d84-9b67-5d13c1ef390e admin admin] PATH_INFO: `/v3/users/5055de19b9b34875809f37665612cdfb` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:49.109391 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bc02ae-954c-4d84-9b67-5d13c1ef390e admin admin] RBAC: Authorizing `identity:delete_user(user_id=5055de19b9b34875809f37665612cdfb)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:49.112161 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bc02ae-954c-4d84-9b67-5d13c1ef390e admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:49.113564 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-0b453d93-d26c-47e4-929a-85140618b1c9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '91f71abcbc54454da9a52debf048ff0f'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:49.116516 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-0b453d93-d26c-47e4-929a-85140618b1c9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '91f71abcbc54454da9a52debf048ff0f'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:49.134573 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-0b453d93-d26c-47e4-929a-85140618b1c9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '91f71abcbc54454da9a52debf048ff0f'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:49.135789 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-0b453d93-d26c-47e4-929a-85140618b1c9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '91f71abcbc54454da9a52debf048ff0f'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:49.158071 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1024/2051] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 01:23:48 2022] DELETE /identity/v3/users/91f71abcbc54454da9a52debf048ff0f => generated 0 bytes in 553 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 01:23:49.159343 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8339d6-7590-47e6-89d8-f2be73f7936c None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:49.159638 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8339d6-7590-47e6-89d8-f2be73f7936c None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:49.159956 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8339d6-7590-47e6-89d8-f2be73f7936c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:49.186728 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-4d8339d6-7590-47e6-89d8-f2be73f7936c None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:23:49.188219 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-4d8339d6-7590-47e6-89d8-f2be73f7936c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:23:49.273426 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1025/2052] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:23:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:23:49.282619 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2b743b-1f33-44bc-9bbb-bbd0f23ed3ba None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:49.282902 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2b743b-1f33-44bc-9bbb-bbd0f23ed3ba None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:49.283249 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2b743b-1f33-44bc-9bbb-bbd0f23ed3ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:49.316005 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-3c2b743b-1f33-44bc-9bbb-bbd0f23ed3ba None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:23:49.318335 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-3c2b743b-1f33-44bc-9bbb-bbd0f23ed3ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:23:49.331315 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1026/2053] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:23:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:23:49.335245 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-a3bc02ae-954c-4d84-9b67-5d13c1ef390e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5055de19b9b34875809f37665612cdfb'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:49.335994 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2b743b-1f33-44bc-9bbb-bbd0f23ed3ba None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:49.345968 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2b743b-1f33-44bc-9bbb-bbd0f23ed3ba None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:49.351499 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0ffa6d-2f07-40ff-8f0f-4a5ca65b24bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:49.352241 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0ffa6d-2f07-40ff-8f0f-4a5ca65b24bd admin admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:49.352628 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0ffa6d-2f07-40ff-8f0f-4a5ca65b24bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:49.353036 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0ffa6d-2f07-40ff-8f0f-4a5ca65b24bd admin admin] PATH_INFO: `/v3/domains` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:49.353789 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0ffa6d-2f07-40ff-8f0f-4a5ca65b24bd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:23:49.354199 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0ffa6d-2f07-40ff-8f0f-4a5ca65b24bd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:49.356102 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0ffa6d-2f07-40ff-8f0f-4a5ca65b24bd admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:49.358630 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-a3bc02ae-954c-4d84-9b67-5d13c1ef390e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5055de19b9b34875809f37665612cdfb'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:49.361183 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-a3bc02ae-954c-4d84-9b67-5d13c1ef390e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5055de19b9b34875809f37665612cdfb'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:49.369856 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1027/2054] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Aug 31 01:23:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:23:49.371778 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-a3bc02ae-954c-4d84-9b67-5d13c1ef390e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5055de19b9b34875809f37665612cdfb'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:49.374924 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0ffa6d-2f07-40ff-8f0f-4a5ca65b24bd admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:49.379611 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-ab0ffa6d-2f07-40ff-8f0f-4a5ca65b24bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:23:49.392609 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0ffa6d-2f07-40ff-8f0f-4a5ca65b24bd admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:49.397456 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b818f8a1-08c3-4465-a468-8ccd6472eb68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:49.398240 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b818f8a1-08c3-4465-a468-8ccd6472eb68 admin admin] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:49.398596 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b818f8a1-08c3-4465-a468-8ccd6472eb68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:49.398940 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b818f8a1-08c3-4465-a468-8ccd6472eb68 admin admin] PATH_INFO: `/v3/projects` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:49.399639 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b818f8a1-08c3-4465-a468-8ccd6472eb68 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:49.401716 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b818f8a1-08c3-4465-a468-8ccd6472eb68 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:49.499574 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-a3bc02ae-954c-4d84-9b67-5d13c1ef390e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5055de19b9b34875809f37665612cdfb'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:49.506413 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1028/2055] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 01:23:49 2022] DELETE /identity/v3/users/5055de19b9b34875809f37665612cdfb => generated 0 bytes in 485 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:23:49.510837 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373c9109-042a-4324-af06-a47a2e26fb94 None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:49.511133 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373c9109-042a-4324-af06-a47a2e26fb94 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:49.511415 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373c9109-042a-4324-af06-a47a2e26fb94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:49.536390 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-373c9109-042a-4324-af06-a47a2e26fb94 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:23:49.541443 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-373c9109-042a-4324-af06-a47a2e26fb94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:23:49.544476 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1028/2056] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 01:23:49 2022] POST /identity/v3/projects => generated 377 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:23:49.548048 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b818f8a1-08c3-4465-a468-8ccd6472eb68 admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:49.550397 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-b818f8a1-08c3-4465-a468-8ccd6472eb68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:23:49.593104 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:23:49.660887 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1029/2057] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:23:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:23:49.662481 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373c9109-042a-4324-af06-a47a2e26fb94 None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:49.671528 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b818f8a1-08c3-4465-a468-8ccd6472eb68 admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:49.677945 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a45d05-fa90-4fc0-a83c-70a434b12a4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:49.678613 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a45d05-fa90-4fc0-a83c-70a434b12a4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:49.678729 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373c9109-042a-4324-af06-a47a2e26fb94 None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:49.678897 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a45d05-fa90-4fc0-a83c-70a434b12a4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:49.679176 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a45d05-fa90-4fc0-a83c-70a434b12a4a admin admin] PATH_INFO: `/v3/projects/1a4173ddac70420586849aa47642a9a3` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:49.680534 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a45d05-fa90-4fc0-a83c-70a434b12a4a admin admin] RBAC: Authorizing `identity:delete_project(project_id=1a4173ddac70420586849aa47642a9a3)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:49.682854 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a45d05-fa90-4fc0-a83c-70a434b12a4a admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:49.683671 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e7e865-0b0c-4894-99db-4c3cb490e145 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:49.684347 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e7e865-0b0c-4894-99db-4c3cb490e145 admin admin] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:49.684883 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e7e865-0b0c-4894-99db-4c3cb490e145 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:49.685354 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e7e865-0b0c-4894-99db-4c3cb490e145 admin admin] PATH_INFO: `/v3/users` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:49.686165 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e7e865-0b0c-4894-99db-4c3cb490e145 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:49.688836 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e7e865-0b0c-4894-99db-4c3cb490e145 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:49.868259 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-04a45d05-fa90-4fc0-a83c-70a434b12a4a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1a4173ddac70420586849aa47642a9a3', 'request_id': 'req-04a45d05-fa90-4fc0-a83c-70a434b12a4a'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:49.880270 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1030/2058] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 01:23:49 2022] POST /identity/v3/users => generated 289 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:23:49.881177 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e7e865-0b0c-4894-99db-4c3cb490e145 admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:49.884658 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-57e7e865-0b0c-4894-99db-4c3cb490e145 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:23:49.889018 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-04a45d05-fa90-4fc0-a83c-70a434b12a4a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1a4173ddac70420586849aa47642a9a3', 'request_id': 'req-04a45d05-fa90-4fc0-a83c-70a434b12a4a'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:49.898838 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-04a45d05-fa90-4fc0-a83c-70a434b12a4a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1a4173ddac70420586849aa47642a9a3', 'request_id': 'req-04a45d05-fa90-4fc0-a83c-70a434b12a4a'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:49.971525 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e7e865-0b0c-4894-99db-4c3cb490e145 admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:49.991140 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-04a45d05-fa90-4fc0-a83c-70a434b12a4a admin admin] The token cache is being invalidate because project 1a4173ddac70420586849aa47642a9a3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81749) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 01:23:49.991684 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-04a45d05-fa90-4fc0-a83c-70a434b12a4a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:49.993197 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-04a45d05-fa90-4fc0-a83c-70a434b12a4a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:49.995202 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1029/2059] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 01:23:49 2022] DELETE /identity/v3/projects/1a4173ddac70420586849aa47642a9a3 => generated 0 bytes in 451 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:23:49.996196 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a45d05-fa90-4fc0-a83c-70a434b12a4a admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:50.011264 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-04a45d05-fa90-4fc0-a83c-70a434b12a4a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:23:50.014395 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:23:50.100135 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6a26cf-2bad-4f20-8f8e-5758c7fde0ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0817c3320d5d41d8bba0550c8ae8dd60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:50.102239 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6a26cf-2bad-4f20-8f8e-5758c7fde0ea service neutron] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:50.102239 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6a26cf-2bad-4f20-8f8e-5758c7fde0ea service neutron] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:50.102239 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6a26cf-2bad-4f20-8f8e-5758c7fde0ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:50.104099 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-4a6a26cf-2bad-4f20-8f8e-5758c7fde0ea service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:23:50.155464 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a45d05-fa90-4fc0-a83c-70a434b12a4a admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:50.162292 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59b8ca2-6ea8-43c2-b95e-6b9dc82f2470 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:50.163044 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59b8ca2-6ea8-43c2-b95e-6b9dc82f2470 admin admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:50.163399 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59b8ca2-6ea8-43c2-b95e-6b9dc82f2470 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:50.163810 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59b8ca2-6ea8-43c2-b95e-6b9dc82f2470 admin admin] PATH_INFO: `/v3/roles` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:50.164605 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59b8ca2-6ea8-43c2-b95e-6b9dc82f2470 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:50.166706 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59b8ca2-6ea8-43c2-b95e-6b9dc82f2470 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:50.188678 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1030/2060] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:23:49 2022] GET /identity/v3/roles => generated 1420 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:23:50.189832 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59b8ca2-6ea8-43c2-b95e-6b9dc82f2470 admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:50.193286 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-d59b8ca2-6ea8-43c2-b95e-6b9dc82f2470 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:23:50.211533 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59b8ca2-6ea8-43c2-b95e-6b9dc82f2470 admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:50.217026 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6d572f-c82d-4ca7-bbf0-2d43781727d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:50.218017 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6d572f-c82d-4ca7-bbf0-2d43781727d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:50.218310 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6d572f-c82d-4ca7-bbf0-2d43781727d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:50.218587 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6d572f-c82d-4ca7-bbf0-2d43781727d7 admin admin] PATH_INFO: `/v3/users/fbdd2d19767648d78f4bd33dbc03ddd8` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:50.219819 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6d572f-c82d-4ca7-bbf0-2d43781727d7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fbdd2d19767648d78f4bd33dbc03ddd8)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:50.221974 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6d572f-c82d-4ca7-bbf0-2d43781727d7 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:50.242444 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6a26cf-2bad-4f20-8f8e-5758c7fde0ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:50.243781 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6a26cf-2bad-4f20-8f8e-5758c7fde0ea service neutron] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:50.256351 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1031/2061] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:23:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:23:50.257122 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6a26cf-2bad-4f20-8f8e-5758c7fde0ea service neutron] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:50.267546 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6a26cf-2bad-4f20-8f8e-5758c7fde0ea service neutron] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:50.273079 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302ba764-4b0e-464b-92d8-4ff9fde14272 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:50.273712 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302ba764-4b0e-464b-92d8-4ff9fde14272 admin admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:50.274110 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302ba764-4b0e-464b-92d8-4ff9fde14272 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:50.274239 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302ba764-4b0e-464b-92d8-4ff9fde14272 admin admin] PATH_INFO: `/v3/roles` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:50.274886 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302ba764-4b0e-464b-92d8-4ff9fde14272 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:50.276499 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302ba764-4b0e-464b-92d8-4ff9fde14272 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:50.287109 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1032/2062] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:23:50 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:23:50.292087 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a3bba2-624b-417d-8939-1d98b1b94a0c None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:50.292502 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a3bba2-624b-417d-8939-1d98b1b94a0c None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:50.292682 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a3bba2-624b-417d-8939-1d98b1b94a0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:50.313439 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-72a3bba2-624b-417d-8939-1d98b1b94a0c None None] MFA Rules not processed for user `1cc3db55df284cd39f28fcfefc619fda`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:23:50.314757 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-72a3bba2-624b-417d-8939-1d98b1b94a0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:23:50.379309 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1033/2063] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:23:50 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:23:50.380406 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a3bba2-624b-417d-8939-1d98b1b94a0c None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:50.391787 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a3bba2-624b-417d-8939-1d98b1b94a0c None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:50.397102 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60adca57-0233-4611-951e-73c8b2134a08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:50.397771 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60adca57-0233-4611-951e-73c8b2134a08 admin admin] REQUEST_METHOD: `PUT` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:50.398065 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60adca57-0233-4611-951e-73c8b2134a08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:50.398340 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60adca57-0233-4611-951e-73c8b2134a08 admin admin] PATH_INFO: `/v3/projects/cd203cd3bc9f472ea6e7553ee837c677/users/9755cee910ff49a6ad059a9cb02989e6/roles/d0b9768f4d014766b9326387f2e11f34` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:50.417048 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60adca57-0233-4611-951e-73c8b2134a08 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd203cd3bc9f472ea6e7553ee837c677, user_id=9755cee910ff49a6ad059a9cb02989e6, role_id=d0b9768f4d014766b9326387f2e11f34)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:50.419727 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60adca57-0233-4611-951e-73c8b2134a08 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:50.502714 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-fc6d572f-c82d-4ca7-bbf0-2d43781727d7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fbdd2d19767648d78f4bd33dbc03ddd8'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:50.582346 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1034/2064] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 01:23:50 2022] PUT /identity/v3/projects/cd203cd3bc9f472ea6e7553ee837c677/users/9755cee910ff49a6ad059a9cb02989e6/roles/d0b9768f4d014766b9326387f2e11f34 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:23:50.582346 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d78a0b2-a4a0-42ea-8ef2-03351b359041 None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:50.582346 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d78a0b2-a4a0-42ea-8ef2-03351b359041 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:50.582346 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d78a0b2-a4a0-42ea-8ef2-03351b359041 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:50.602428 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-9d78a0b2-a4a0-42ea-8ef2-03351b359041 None None] MFA Rules not processed for user `9755cee910ff49a6ad059a9cb02989e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:23:50.603665 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-9d78a0b2-a4a0-42ea-8ef2-03351b359041 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:23:50.608395 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-fc6d572f-c82d-4ca7-bbf0-2d43781727d7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fbdd2d19767648d78f4bd33dbc03ddd8'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:50.609155 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-fc6d572f-c82d-4ca7-bbf0-2d43781727d7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fbdd2d19767648d78f4bd33dbc03ddd8'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:50.631347 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-fc6d572f-c82d-4ca7-bbf0-2d43781727d7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fbdd2d19767648d78f4bd33dbc03ddd8'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:50.633451 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-fc6d572f-c82d-4ca7-bbf0-2d43781727d7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fbdd2d19767648d78f4bd33dbc03ddd8'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:50.672572 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1031/2065] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 01:23:50 2022] DELETE /identity/v3/users/fbdd2d19767648d78f4bd33dbc03ddd8 => generated 0 bytes in 483 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:23:50.731860 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1035/2066] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:23:50 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 01:23:50.734938 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6d572f-c82d-4ca7-bbf0-2d43781727d7 admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:50.736325 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-fc6d572f-c82d-4ca7-bbf0-2d43781727d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:23:50.881309 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6d572f-c82d-4ca7-bbf0-2d43781727d7 admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:50.889758 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e5e147-6e2f-4d35-a271-e76603e88d2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:50.890861 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e5e147-6e2f-4d35-a271-e76603e88d2d admin admin] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:50.891402 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e5e147-6e2f-4d35-a271-e76603e88d2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:50.891868 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e5e147-6e2f-4d35-a271-e76603e88d2d admin admin] PATH_INFO: `/v3/projects` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:50.893151 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e5e147-6e2f-4d35-a271-e76603e88d2d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:50.896305 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e5e147-6e2f-4d35-a271-e76603e88d2d admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:50.956266 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d78a0b2-a4a0-42ea-8ef2-03351b359041 None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:50.958548 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-9d78a0b2-a4a0-42ea-8ef2-03351b359041 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:23:51.055778 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1032/2067] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 01:23:50 2022] POST /identity/v3/projects => generated 379 bytes in 321 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:23:51.056759 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e5e147-6e2f-4d35-a271-e76603e88d2d admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:51.057054 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d78a0b2-a4a0-42ea-8ef2-03351b359041 None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:51.148462 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec964d2-02dd-4ea5-a622-2500d64aa3bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:51.149192 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec964d2-02dd-4ea5-a622-2500d64aa3bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:51.149569 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec964d2-02dd-4ea5-a622-2500d64aa3bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:51.149862 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec964d2-02dd-4ea5-a622-2500d64aa3bb admin admin] PATH_INFO: `/v3/projects/64a2333f399f41d693016c4fd9f41af5` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:51.151355 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec964d2-02dd-4ea5-a622-2500d64aa3bb admin admin] RBAC: Authorizing `identity:delete_project(project_id=64a2333f399f41d693016c4fd9f41af5)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:51.153469 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec964d2-02dd-4ea5-a622-2500d64aa3bb admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:51.173473 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e5e147-6e2f-4d35-a271-e76603e88d2d admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:51.179790 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c116545-7c9a-48cd-8f06-66aba0cede58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:51.180519 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c116545-7c9a-48cd-8f06-66aba0cede58 admin admin] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:51.180907 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c116545-7c9a-48cd-8f06-66aba0cede58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:51.181257 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c116545-7c9a-48cd-8f06-66aba0cede58 admin admin] PATH_INFO: `/v3/users` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:51.182214 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c116545-7c9a-48cd-8f06-66aba0cede58 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:51.184355 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c116545-7c9a-48cd-8f06-66aba0cede58 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:51.293434 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1033/2068] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 01:23:51 2022] POST /identity/v3/users => generated 290 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:23:51.293434 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c116545-7c9a-48cd-8f06-66aba0cede58 admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:51.293899 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-5c116545-7c9a-48cd-8f06-66aba0cede58 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:23:51.302336 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-dec964d2-02dd-4ea5-a622-2500d64aa3bb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '64a2333f399f41d693016c4fd9f41af5', 'request_id': 'req-dec964d2-02dd-4ea5-a622-2500d64aa3bb'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:51.308535 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c116545-7c9a-48cd-8f06-66aba0cede58 admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:51.309349 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-dec964d2-02dd-4ea5-a622-2500d64aa3bb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '64a2333f399f41d693016c4fd9f41af5', 'request_id': 'req-dec964d2-02dd-4ea5-a622-2500d64aa3bb'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:51.372835 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:23:51.378314 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee1f59b-4c7a-47ec-9a17-78126b1bf1b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:51.379722 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee1f59b-4c7a-47ec-9a17-78126b1bf1b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:51.380063 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee1f59b-4c7a-47ec-9a17-78126b1bf1b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:51.380347 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee1f59b-4c7a-47ec-9a17-78126b1bf1b5 admin admin] PATH_INFO: `/v3/projects/8fa93a330f004af585babdddf27ab6dc` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:51.381506 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee1f59b-4c7a-47ec-9a17-78126b1bf1b5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8fa93a330f004af585babdddf27ab6dc)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:51.383525 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee1f59b-4c7a-47ec-9a17-78126b1bf1b5 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:51.407656 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-dec964d2-02dd-4ea5-a622-2500d64aa3bb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '64a2333f399f41d693016c4fd9f41af5', 'request_id': 'req-dec964d2-02dd-4ea5-a622-2500d64aa3bb'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:51.423753 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-dec964d2-02dd-4ea5-a622-2500d64aa3bb admin admin] The token cache is being invalidate because project 64a2333f399f41d693016c4fd9f41af5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81750) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 01:23:51.424051 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-dec964d2-02dd-4ea5-a622-2500d64aa3bb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:51.424921 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-dec964d2-02dd-4ea5-a622-2500d64aa3bb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:51.428956 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1036/2069] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 01:23:50 2022] DELETE /identity/v3/projects/64a2333f399f41d693016c4fd9f41af5 => generated 0 bytes in 471 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:23:51.429892 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec964d2-02dd-4ea5-a622-2500d64aa3bb admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:51.431408 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-dec964d2-02dd-4ea5-a622-2500d64aa3bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:23:51.445119 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec964d2-02dd-4ea5-a622-2500d64aa3bb admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:51.450835 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d101c73a-23c1-475d-a5fb-c7c1b1512d40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:51.451317 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d101c73a-23c1-475d-a5fb-c7c1b1512d40 admin admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:51.451600 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d101c73a-23c1-475d-a5fb-c7c1b1512d40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:51.451876 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d101c73a-23c1-475d-a5fb-c7c1b1512d40 admin admin] PATH_INFO: `/v3/roles` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:51.452527 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d101c73a-23c1-475d-a5fb-c7c1b1512d40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:51.454584 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d101c73a-23c1-475d-a5fb-c7c1b1512d40 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:51.466970 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1037/2070] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:23:51 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:23:51.467889 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d101c73a-23c1-475d-a5fb-c7c1b1512d40 admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:51.471287 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-d101c73a-23c1-475d-a5fb-c7c1b1512d40 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:23:51.498512 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d101c73a-23c1-475d-a5fb-c7c1b1512d40 admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:51.511434 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-8ee1f59b-4c7a-47ec-9a17-78126b1bf1b5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8fa93a330f004af585babdddf27ab6dc', 'request_id': 'req-8ee1f59b-4c7a-47ec-9a17-78126b1bf1b5'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:51.517353 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-8ee1f59b-4c7a-47ec-9a17-78126b1bf1b5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8fa93a330f004af585babdddf27ab6dc', 'request_id': 'req-8ee1f59b-4c7a-47ec-9a17-78126b1bf1b5'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:51.524574 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-8ee1f59b-4c7a-47ec-9a17-78126b1bf1b5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8fa93a330f004af585babdddf27ab6dc', 'request_id': 'req-8ee1f59b-4c7a-47ec-9a17-78126b1bf1b5'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:51.560656 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb5da84-e005-4d4c-bd65-1505359df4bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:51.561332 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb5da84-e005-4d4c-bd65-1505359df4bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:51.561616 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb5da84-e005-4d4c-bd65-1505359df4bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:51.561897 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb5da84-e005-4d4c-bd65-1505359df4bc admin admin] PATH_INFO: `/v3/users/3067b984964e41c8ae3f74fb89783241` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:51.563116 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb5da84-e005-4d4c-bd65-1505359df4bc admin admin] RBAC: Authorizing `identity:delete_user(user_id=3067b984964e41c8ae3f74fb89783241)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:51.565100 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb5da84-e005-4d4c-bd65-1505359df4bc admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:51.669136 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-8ee1f59b-4c7a-47ec-9a17-78126b1bf1b5 admin admin] The token cache is being invalidate because project 8fa93a330f004af585babdddf27ab6dc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81749) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 01:23:51.669432 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-8ee1f59b-4c7a-47ec-9a17-78126b1bf1b5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:51.670068 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-8ee1f59b-4c7a-47ec-9a17-78126b1bf1b5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:51.671584 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1034/2071] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 01:23:51 2022] DELETE /identity/v3/projects/8fa93a330f004af585babdddf27ab6dc => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:23:51.672426 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee1f59b-4c7a-47ec-9a17-78126b1bf1b5 admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:51.673751 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-8ee1f59b-4c7a-47ec-9a17-78126b1bf1b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:23:51.686970 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee1f59b-4c7a-47ec-9a17-78126b1bf1b5 admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:51.691777 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b1dd31-b401-4899-9676-1fd593be0708 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:51.692362 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b1dd31-b401-4899-9676-1fd593be0708 admin admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:51.692642 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b1dd31-b401-4899-9676-1fd593be0708 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:51.692954 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b1dd31-b401-4899-9676-1fd593be0708 admin admin] PATH_INFO: `/v3/roles` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:51.693589 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b1dd31-b401-4899-9676-1fd593be0708 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:51.695237 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b1dd31-b401-4899-9676-1fd593be0708 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:51.705267 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1035/2072] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:23:51 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:23:51.706376 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c025e06-a22a-444b-a1de-7e345ba92f57 None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:51.706660 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c025e06-a22a-444b-a1de-7e345ba92f57 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:51.706930 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c025e06-a22a-444b-a1de-7e345ba92f57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:51.725710 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-9c025e06-a22a-444b-a1de-7e345ba92f57 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:23:51.726911 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-9c025e06-a22a-444b-a1de-7e345ba92f57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:23:51.734709 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1036/2073] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:23:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:23:51.736130 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c025e06-a22a-444b-a1de-7e345ba92f57 None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:51.746583 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c025e06-a22a-444b-a1de-7e345ba92f57 None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:51.751376 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0df1838-9300-4dc7-91f5-9f454d370e01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:51.752025 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0df1838-9300-4dc7-91f5-9f454d370e01 admin admin] REQUEST_METHOD: `PUT` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:51.752325 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0df1838-9300-4dc7-91f5-9f454d370e01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:51.752581 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0df1838-9300-4dc7-91f5-9f454d370e01 admin admin] PATH_INFO: `/v3/projects/69c2c084da1c493d9e4747639e68739f/users/4e18f7aded084c76a4580dfaea91d397/roles/d0b9768f4d014766b9326387f2e11f34` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:51.766980 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0df1838-9300-4dc7-91f5-9f454d370e01 admin admin] RBAC: Authorizing `identity:create_grant(project_id=69c2c084da1c493d9e4747639e68739f, user_id=4e18f7aded084c76a4580dfaea91d397, role_id=d0b9768f4d014766b9326387f2e11f34)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:51.769885 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0df1838-9300-4dc7-91f5-9f454d370e01 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:51.821230 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-7fb5da84-e005-4d4c-bd65-1505359df4bc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3067b984964e41c8ae3f74fb89783241'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:51.832825 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-7fb5da84-e005-4d4c-bd65-1505359df4bc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3067b984964e41c8ae3f74fb89783241'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:51.833553 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-7fb5da84-e005-4d4c-bd65-1505359df4bc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3067b984964e41c8ae3f74fb89783241'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:51.840408 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-7fb5da84-e005-4d4c-bd65-1505359df4bc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3067b984964e41c8ae3f74fb89783241'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:51.911211 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-7fb5da84-e005-4d4c-bd65-1505359df4bc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3067b984964e41c8ae3f74fb89783241'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:51.917961 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1037/2074] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 01:23:51 2022] PUT /identity/v3/projects/69c2c084da1c493d9e4747639e68739f/users/4e18f7aded084c76a4580dfaea91d397/roles/d0b9768f4d014766b9326387f2e11f34 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:23:51.919329 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1038/2075] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 01:23:51 2022] DELETE /identity/v3/users/3067b984964e41c8ae3f74fb89783241 => generated 0 bytes in 452 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:23:51.919677 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1eb372-b91c-4fb6-be8d-e7ab8916505a None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:51.920189 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1eb372-b91c-4fb6-be8d-e7ab8916505a None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:51.920631 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1eb372-b91c-4fb6-be8d-e7ab8916505a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:51.931237 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d96922-2162-4b6c-b0b6-f5466b2a7091 None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:51.931633 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d96922-2162-4b6c-b0b6-f5466b2a7091 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:51.931841 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d96922-2162-4b6c-b0b6-f5466b2a7091 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:51.947974 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-1e1eb372-b91c-4fb6-be8d-e7ab8916505a None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:23:51.949311 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-1e1eb372-b91c-4fb6-be8d-e7ab8916505a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:23:51.963077 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-d4d96922-2162-4b6c-b0b6-f5466b2a7091 None None] MFA Rules not processed for user `4e18f7aded084c76a4580dfaea91d397`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:23:51.964564 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-d4d96922-2162-4b6c-b0b6-f5466b2a7091 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:23:52.021210 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1038/2076] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:23:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:23:52.024751 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1eb372-b91c-4fb6-be8d-e7ab8916505a None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:52.035831 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1eb372-b91c-4fb6-be8d-e7ab8916505a None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:52.039459 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1039/2077] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:23:51 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:23:52.041020 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802745c4-fd46-414e-b7ed-1cb43bd17c30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:52.048384 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802745c4-fd46-414e-b7ed-1cb43bd17c30 admin admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:52.048384 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802745c4-fd46-414e-b7ed-1cb43bd17c30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:52.048384 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802745c4-fd46-414e-b7ed-1cb43bd17c30 admin admin] PATH_INFO: `/v3/domains` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:52.048384 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802745c4-fd46-414e-b7ed-1cb43bd17c30 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:23:52.048384 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802745c4-fd46-414e-b7ed-1cb43bd17c30 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:52.048384 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802745c4-fd46-414e-b7ed-1cb43bd17c30 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:52.049694 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d96922-2162-4b6c-b0b6-f5466b2a7091 None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:52.051315 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-d4d96922-2162-4b6c-b0b6-f5466b2a7091 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:23:52.060151 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1039/2078] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Aug 31 01:23:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 01:23:52.061173 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802745c4-fd46-414e-b7ed-1cb43bd17c30 admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:52.066583 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d96922-2162-4b6c-b0b6-f5466b2a7091 None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:52.075118 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802745c4-fd46-414e-b7ed-1cb43bd17c30 admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:52.075277 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95441d9b-967c-4844-a636-cbbaa7c95564 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:52.075277 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95441d9b-967c-4844-a636-cbbaa7c95564 admin admin] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:52.075277 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95441d9b-967c-4844-a636-cbbaa7c95564 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:52.075277 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95441d9b-967c-4844-a636-cbbaa7c95564 admin admin] PATH_INFO: `/v3/projects` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:52.075641 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95441d9b-967c-4844-a636-cbbaa7c95564 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:52.075912 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95441d9b-967c-4844-a636-cbbaa7c95564 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:52.078371 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4962fa-6b1b-40d1-8788-80d6f86852e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:52.079244 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4962fa-6b1b-40d1-8788-80d6f86852e4 admin admin] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:52.079528 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4962fa-6b1b-40d1-8788-80d6f86852e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:52.080321 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4962fa-6b1b-40d1-8788-80d6f86852e4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:52.080445 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4962fa-6b1b-40d1-8788-80d6f86852e4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:52.082970 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4962fa-6b1b-40d1-8788-80d6f86852e4 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:52.207125 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1040/2079] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 01:23:52 2022] POST /identity/v3/projects => generated 379 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 01:23:52.209401 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95441d9b-967c-4844-a636-cbbaa7c95564 admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:52.223395 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1040/2080] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 01:23:52 2022] POST /identity/v3/projects => generated 375 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 01:23:52.224324 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4962fa-6b1b-40d1-8788-80d6f86852e4 admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:52.284325 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95441d9b-967c-4844-a636-cbbaa7c95564 admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:52.299526 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4962fa-6b1b-40d1-8788-80d6f86852e4 admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:52.306806 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc529546-7a28-4f08-8a9b-b69c8d5e4ac0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:52.307590 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc529546-7a28-4f08-8a9b-b69c8d5e4ac0 admin admin] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:52.307737 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc529546-7a28-4f08-8a9b-b69c8d5e4ac0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:52.307904 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc529546-7a28-4f08-8a9b-b69c8d5e4ac0 admin admin] PATH_INFO: `/v3/users` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:52.308540 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc529546-7a28-4f08-8a9b-b69c8d5e4ac0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:52.310538 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc529546-7a28-4f08-8a9b-b69c8d5e4ac0 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:52.349288 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad77a5c-c331-4a39-a887-534323d75448 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:52.349931 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad77a5c-c331-4a39-a887-534323d75448 admin admin] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:52.350266 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad77a5c-c331-4a39-a887-534323d75448 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:52.350558 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad77a5c-c331-4a39-a887-534323d75448 admin admin] PATH_INFO: `/v3/users` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:52.351165 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad77a5c-c331-4a39-a887-534323d75448 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:52.353159 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad77a5c-c331-4a39-a887-534323d75448 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:52.414419 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:23:52.421221 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1041/2081] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 01:23:52 2022] POST /identity/v3/users => generated 288 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:23:52.421221 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc529546-7a28-4f08-8a9b-b69c8d5e4ac0 admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:52.424418 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-dc529546-7a28-4f08-8a9b-b69c8d5e4ac0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:23:52.435998 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc529546-7a28-4f08-8a9b-b69c8d5e4ac0 admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:52.442274 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3560ba6d-2e9b-4207-a168-f87ef9d941e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:52.442888 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3560ba6d-2e9b-4207-a168-f87ef9d941e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:52.443183 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3560ba6d-2e9b-4207-a168-f87ef9d941e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:52.443442 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3560ba6d-2e9b-4207-a168-f87ef9d941e7 admin admin] PATH_INFO: `/v3/projects/c9ef310bc8e046d9bea9e36bc6573c72` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:52.445091 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3560ba6d-2e9b-4207-a168-f87ef9d941e7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c9ef310bc8e046d9bea9e36bc6573c72)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:52.447351 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3560ba6d-2e9b-4207-a168-f87ef9d941e7 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:52.480582 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1041/2082] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 01:23:52 2022] POST /identity/v3/users => generated 296 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 01:23:52.480582 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad77a5c-c331-4a39-a887-534323d75448 admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:52.489403 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad77a5c-c331-4a39-a887-534323d75448 admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:52.494436 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d10b5e-e86f-479d-90ed-55a8dba74469 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:52.495056 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d10b5e-e86f-479d-90ed-55a8dba74469 admin admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:52.495340 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d10b5e-e86f-479d-90ed-55a8dba74469 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:52.495619 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d10b5e-e86f-479d-90ed-55a8dba74469 admin admin] PATH_INFO: `/v3/roles` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:52.496274 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d10b5e-e86f-479d-90ed-55a8dba74469 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:52.497959 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d10b5e-e86f-479d-90ed-55a8dba74469 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:52.512015 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1042/2083] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:23:52 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:23:52.512015 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d10b5e-e86f-479d-90ed-55a8dba74469 admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:52.521184 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d10b5e-e86f-479d-90ed-55a8dba74469 admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:52.527596 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf22699-691c-4bf3-87be-a04098647f26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:52.528751 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf22699-691c-4bf3-87be-a04098647f26 admin admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:52.528751 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf22699-691c-4bf3-87be-a04098647f26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:52.528957 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf22699-691c-4bf3-87be-a04098647f26 admin admin] PATH_INFO: `/v3/roles` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:52.529424 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf22699-691c-4bf3-87be-a04098647f26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:52.531155 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf22699-691c-4bf3-87be-a04098647f26 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:52.544995 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1043/2084] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:23:52 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:23:52.544995 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf22699-691c-4bf3-87be-a04098647f26 admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:52.553173 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf22699-691c-4bf3-87be-a04098647f26 admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:52.557806 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66893ee8-a410-4298-85e1-7164d8a38e61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:52.558394 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66893ee8-a410-4298-85e1-7164d8a38e61 admin admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:52.558675 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66893ee8-a410-4298-85e1-7164d8a38e61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:52.558949 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66893ee8-a410-4298-85e1-7164d8a38e61 admin admin] PATH_INFO: `/v3/roles` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:52.559577 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66893ee8-a410-4298-85e1-7164d8a38e61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:52.561319 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66893ee8-a410-4298-85e1-7164d8a38e61 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:52.582566 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1044/2085] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:23:52 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:23:52.582566 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66893ee8-a410-4298-85e1-7164d8a38e61 admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:52.582566 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-66893ee8-a410-4298-85e1-7164d8a38e61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:23:52.592824 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66893ee8-a410-4298-85e1-7164d8a38e61 admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:52.608084 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-3560ba6d-2e9b-4207-a168-f87ef9d941e7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c9ef310bc8e046d9bea9e36bc6573c72', 'request_id': 'req-3560ba6d-2e9b-4207-a168-f87ef9d941e7'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:52.614944 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-3560ba6d-2e9b-4207-a168-f87ef9d941e7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c9ef310bc8e046d9bea9e36bc6573c72', 'request_id': 'req-3560ba6d-2e9b-4207-a168-f87ef9d941e7'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:52.622189 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-3560ba6d-2e9b-4207-a168-f87ef9d941e7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c9ef310bc8e046d9bea9e36bc6573c72', 'request_id': 'req-3560ba6d-2e9b-4207-a168-f87ef9d941e7'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:52.657240 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c2ba57-bccf-4b33-9985-87be7318d245 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:52.657938 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c2ba57-bccf-4b33-9985-87be7318d245 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:52.658153 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c2ba57-bccf-4b33-9985-87be7318d245 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:52.658678 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c2ba57-bccf-4b33-9985-87be7318d245 admin admin] PATH_INFO: `/v3/users/a69eca1946824a148db97ae7f7a67eb8` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:52.659776 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c2ba57-bccf-4b33-9985-87be7318d245 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a69eca1946824a148db97ae7f7a67eb8)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:52.661742 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c2ba57-bccf-4b33-9985-87be7318d245 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:52.709313 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-3560ba6d-2e9b-4207-a168-f87ef9d941e7 admin admin] The token cache is being invalidate because project c9ef310bc8e046d9bea9e36bc6573c72 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81749) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 01:23:52.709812 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-3560ba6d-2e9b-4207-a168-f87ef9d941e7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:52.710478 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-3560ba6d-2e9b-4207-a168-f87ef9d941e7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:52.712130 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1042/2086] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 01:23:52 2022] DELETE /identity/v3/projects/c9ef310bc8e046d9bea9e36bc6573c72 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:23:52.713069 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3560ba6d-2e9b-4207-a168-f87ef9d941e7 admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:52.714541 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-3560ba6d-2e9b-4207-a168-f87ef9d941e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:23:52.730422 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3560ba6d-2e9b-4207-a168-f87ef9d941e7 admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:52.735004 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d51641-f010-4160-b3c5-869bd2614d43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:52.735664 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d51641-f010-4160-b3c5-869bd2614d43 admin admin] REQUEST_METHOD: `PUT` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:52.735966 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d51641-f010-4160-b3c5-869bd2614d43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:52.736243 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d51641-f010-4160-b3c5-869bd2614d43 admin admin] PATH_INFO: `/v3/projects/7d4a7299781446f998ddeb413556d2b6/users/58ec6f35f3d94f03846a171a050f13a6/roles/d009411496b8452b88e77ffb8db015a7` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:52.751996 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d51641-f010-4160-b3c5-869bd2614d43 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d4a7299781446f998ddeb413556d2b6, user_id=58ec6f35f3d94f03846a171a050f13a6, role_id=d009411496b8452b88e77ffb8db015a7)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:52.756246 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d51641-f010-4160-b3c5-869bd2614d43 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:52.883040 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:23:52.889780 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1043/2087] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 01:23:52 2022] PUT /identity/v3/projects/7d4a7299781446f998ddeb413556d2b6/users/58ec6f35f3d94f03846a171a050f13a6/roles/d009411496b8452b88e77ffb8db015a7 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:23:52.890593 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d51641-f010-4160-b3c5-869bd2614d43 admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:52.892270 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-67d51641-f010-4160-b3c5-869bd2614d43 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:23:52.962538 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d51641-f010-4160-b3c5-869bd2614d43 admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:52.967272 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0485eea1-2e08-47c1-a689-f8cb52d990b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:52.967933 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0485eea1-2e08-47c1-a689-f8cb52d990b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:52.968219 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0485eea1-2e08-47c1-a689-f8cb52d990b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:52.968498 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0485eea1-2e08-47c1-a689-f8cb52d990b5 admin admin] PATH_INFO: `/v3/projects/db5c6269a518420088c3293504575903/users/c58de488767a410a9d7de3a541fc8d88/roles/d0b9768f4d014766b9326387f2e11f34` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:52.983006 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0485eea1-2e08-47c1-a689-f8cb52d990b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=db5c6269a518420088c3293504575903, user_id=c58de488767a410a9d7de3a541fc8d88, role_id=d0b9768f4d014766b9326387f2e11f34)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:52.985798 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0485eea1-2e08-47c1-a689-f8cb52d990b5 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:52.992619 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-60c2ba57-bccf-4b33-9985-87be7318d245 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a69eca1946824a148db97ae7f7a67eb8'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:53.005027 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-60c2ba57-bccf-4b33-9985-87be7318d245 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a69eca1946824a148db97ae7f7a67eb8'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:53.005723 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-60c2ba57-bccf-4b33-9985-87be7318d245 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a69eca1946824a148db97ae7f7a67eb8'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:53.012567 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-60c2ba57-bccf-4b33-9985-87be7318d245 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a69eca1946824a148db97ae7f7a67eb8'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:53.063865 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1044/2088] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 01:23:52 2022] PUT /identity/v3/projects/db5c6269a518420088c3293504575903/users/c58de488767a410a9d7de3a541fc8d88/roles/d0b9768f4d014766b9326387f2e11f34 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:23:53.065406 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d9d2c8-c251-489d-9292-30e11756a144 None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:53.065693 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d9d2c8-c251-489d-9292-30e11756a144 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:53.066012 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d9d2c8-c251-489d-9292-30e11756a144 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:53.086279 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-00d9d2c8-c251-489d-9292-30e11756a144 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:23:53.087516 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-00d9d2c8-c251-489d-9292-30e11756a144 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:23:53.131654 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-60c2ba57-bccf-4b33-9985-87be7318d245 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a69eca1946824a148db97ae7f7a67eb8'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:53.138967 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1045/2089] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 01:23:52 2022] DELETE /identity/v3/users/a69eca1946824a148db97ae7f7a67eb8 => generated 0 bytes in 565 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:23:53.143277 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c2ba57-bccf-4b33-9985-87be7318d245 admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:53.145368 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-60c2ba57-bccf-4b33-9985-87be7318d245 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:23:53.148225 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1045/2090] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:23:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:23:53.149509 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f783fc52-d1d4-4057-830a-ed2231b28fc5 None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:53.149802 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f783fc52-d1d4-4057-830a-ed2231b28fc5 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:53.150080 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f783fc52-d1d4-4057-830a-ed2231b28fc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:53.181071 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-f783fc52-d1d4-4057-830a-ed2231b28fc5 None None] MFA Rules not processed for user `c58de488767a410a9d7de3a541fc8d88`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:23:53.182134 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-f783fc52-d1d4-4057-830a-ed2231b28fc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:23:53.216736 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c2ba57-bccf-4b33-9985-87be7318d245 admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:53.223522 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b08b38-1053-4cd8-91f5-0184fc10eca0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:53.224264 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b08b38-1053-4cd8-91f5-0184fc10eca0 admin admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:53.224556 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b08b38-1053-4cd8-91f5-0184fc10eca0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:53.224871 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b08b38-1053-4cd8-91f5-0184fc10eca0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:53.225519 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b08b38-1053-4cd8-91f5-0184fc10eca0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:53.227148 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b08b38-1053-4cd8-91f5-0184fc10eca0 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:53.238444 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1046/2091] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:23:53 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:23:53.242206 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fcdb4f-36ee-4dd6-b30d-ffec7eae622e None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:53.242497 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fcdb4f-36ee-4dd6-b30d-ffec7eae622e None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:53.242776 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fcdb4f-36ee-4dd6-b30d-ffec7eae622e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:53.249218 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1046/2092] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:23:53 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:23:53.250578 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5c5482-111b-44b2-bac9-29a885b29429 None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:53.250875 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5c5482-111b-44b2-bac9-29a885b29429 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:53.251156 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5c5482-111b-44b2-bac9-29a885b29429 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:53.262117 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-36fcdb4f-36ee-4dd6-b30d-ffec7eae622e None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:23:53.263464 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-36fcdb4f-36ee-4dd6-b30d-ffec7eae622e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:23:53.271492 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1047/2093] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:23:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:23:53.271626 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-8d5c5482-111b-44b2-bac9-29a885b29429 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:23:53.272102 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fcdb4f-36ee-4dd6-b30d-ffec7eae622e None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:53.272588 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-8d5c5482-111b-44b2-bac9-29a885b29429 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:23:53.282074 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1047/2094] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:23:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:23:53.282217 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fcdb4f-36ee-4dd6-b30d-ffec7eae622e None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:53.282895 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5c5482-111b-44b2-bac9-29a885b29429 None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:53.284183 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-8d5c5482-111b-44b2-bac9-29a885b29429 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:23:53.289967 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef936f91-e6e1-4837-9326-1b6038a004c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:53.290749 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef936f91-e6e1-4837-9326-1b6038a004c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:53.291038 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef936f91-e6e1-4837-9326-1b6038a004c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:53.291312 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef936f91-e6e1-4837-9326-1b6038a004c9 admin admin] PATH_INFO: `/v3/projects/7d4a7299781446f998ddeb413556d2b6/users/58ec6f35f3d94f03846a171a050f13a6/roles/d009411496b8452b88e77ffb8db015a7` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:53.293436 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef936f91-e6e1-4837-9326-1b6038a004c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d4a7299781446f998ddeb413556d2b6, user_id=58ec6f35f3d94f03846a171a050f13a6, role_id=d009411496b8452b88e77ffb8db015a7)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:53.296132 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef936f91-e6e1-4837-9326-1b6038a004c9 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:53.307411 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1048/2095] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 01:23:53 2022] PUT /identity/v3/projects/7d4a7299781446f998ddeb413556d2b6/users/58ec6f35f3d94f03846a171a050f13a6/roles/d009411496b8452b88e77ffb8db015a7 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:23:53.308306 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef936f91-e6e1-4837-9326-1b6038a004c9 admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:53.310029 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-ef936f91-e6e1-4837-9326-1b6038a004c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:23:53.344965 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5c5482-111b-44b2-bac9-29a885b29429 None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:53.368622 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef936f91-e6e1-4837-9326-1b6038a004c9 admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:53.373417 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a48641-89ff-47ec-ad7f-370b43159106 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0817c3320d5d41d8bba0550c8ae8dd60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:53.374142 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a48641-89ff-47ec-ad7f-370b43159106 service neutron] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:53.374520 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a48641-89ff-47ec-ad7f-370b43159106 service neutron] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:53.374893 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a48641-89ff-47ec-ad7f-370b43159106 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:53.395003 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f54d461-c7f7-45cb-8281-06ec22bb7f4b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8590c288028418ebe7a4163fd8039f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:53.395736 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f54d461-c7f7-45cb-8281-06ec22bb7f4b service glance] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:53.396329 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f54d461-c7f7-45cb-8281-06ec22bb7f4b service glance] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:53.396555 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f54d461-c7f7-45cb-8281-06ec22bb7f4b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:53.437842 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a48641-89ff-47ec-ad7f-370b43159106 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:53.439259 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a48641-89ff-47ec-ad7f-370b43159106 service neutron] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:53.450416 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1049/2096] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:23:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:23:53.451264 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a48641-89ff-47ec-ad7f-370b43159106 service neutron] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:53.456763 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f54d461-c7f7-45cb-8281-06ec22bb7f4b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:53.458573 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f54d461-c7f7-45cb-8281-06ec22bb7f4b service glance] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:53.461468 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a48641-89ff-47ec-ad7f-370b43159106 service neutron] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:53.466767 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc94a1f-1405-4f42-8cb5-7d129d58f5ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:53.467513 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc94a1f-1405-4f42-8cb5-7d129d58f5ab admin admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:53.467788 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc94a1f-1405-4f42-8cb5-7d129d58f5ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:53.468065 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc94a1f-1405-4f42-8cb5-7d129d58f5ab admin admin] PATH_INFO: `/v3/domains` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:53.469511 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1048/2097] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:23:53 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:23:53.475409 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc94a1f-1405-4f42-8cb5-7d129d58f5ab admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:23:53.475409 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc94a1f-1405-4f42-8cb5-7d129d58f5ab admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:53.475409 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc94a1f-1405-4f42-8cb5-7d129d58f5ab admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:53.475692 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c614a165-cd86-4f1e-a1d4-4de7561c5093 None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:53.475692 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c614a165-cd86-4f1e-a1d4-4de7561c5093 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:53.475692 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c614a165-cd86-4f1e-a1d4-4de7561c5093 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:53.478295 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:23:53.484868 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1050/2098] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Aug 31 01:23:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:23:53.489695 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc94a1f-1405-4f42-8cb5-7d129d58f5ab admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:53.503148 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc94a1f-1405-4f42-8cb5-7d129d58f5ab admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:53.508091 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef903d3e-6a98-4abc-a9b3-a8853f1449d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:53.508715 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef903d3e-6a98-4abc-a9b3-a8853f1449d0 admin admin] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:53.509058 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef903d3e-6a98-4abc-a9b3-a8853f1449d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:53.509359 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef903d3e-6a98-4abc-a9b3-a8853f1449d0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:53.510008 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef903d3e-6a98-4abc-a9b3-a8853f1449d0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:53.510997 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-c614a165-cd86-4f1e-a1d4-4de7561c5093 None None] MFA Rules not processed for user `58ec6f35f3d94f03846a171a050f13a6`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:23:53.511957 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef903d3e-6a98-4abc-a9b3-a8853f1449d0 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:53.512316 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-c614a165-cd86-4f1e-a1d4-4de7561c5093 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:23:53.611636 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1049/2099] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:23:53 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:23:53.622830 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1051/2100] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 01:23:53 2022] POST /identity/v3/projects => generated 395 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 01:23:53.623505 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c614a165-cd86-4f1e-a1d4-4de7561c5093 None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:53.625411 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef903d3e-6a98-4abc-a9b3-a8853f1449d0 admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:53.629226 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-ef903d3e-6a98-4abc-a9b3-a8853f1449d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:23:53.632765 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c614a165-cd86-4f1e-a1d4-4de7561c5093 None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:53.696632 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef903d3e-6a98-4abc-a9b3-a8853f1449d0 admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:53.715175 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de231cef-c9db-4051-aa5d-9c46f97644c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:53.719170 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de231cef-c9db-4051-aa5d-9c46f97644c3 admin admin] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:53.719286 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f535f85b-54cc-40ae-b1c9-3f3a055359ff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5c8fc69c518412ba1aad61a9f4f558c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:53.719389 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de231cef-c9db-4051-aa5d-9c46f97644c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:53.719389 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de231cef-c9db-4051-aa5d-9c46f97644c3 admin admin] PATH_INFO: `/v3/users` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:53.719601 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f535f85b-54cc-40ae-b1c9-3f3a055359ff service cinder] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:53.719691 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de231cef-c9db-4051-aa5d-9c46f97644c3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:53.720143 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f535f85b-54cc-40ae-b1c9-3f3a055359ff service cinder] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:53.720588 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f535f85b-54cc-40ae-b1c9-3f3a055359ff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:53.721584 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de231cef-c9db-4051-aa5d-9c46f97644c3 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:53.826785 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1050/2101] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 01:23:53 2022] POST /identity/v3/users => generated 298 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:23:53.836223 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f535f85b-54cc-40ae-b1c9-3f3a055359ff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:53.839904 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f535f85b-54cc-40ae-b1c9-3f3a055359ff service cinder] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:53.843113 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de231cef-c9db-4051-aa5d-9c46f97644c3 admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:53.857794 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de231cef-c9db-4051-aa5d-9c46f97644c3 admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:53.869733 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf35c35-7186-4022-8c95-04caa75bad8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:53.870321 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf35c35-7186-4022-8c95-04caa75bad8f admin admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:53.870606 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf35c35-7186-4022-8c95-04caa75bad8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:53.870885 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf35c35-7186-4022-8c95-04caa75bad8f admin admin] PATH_INFO: `/v3/roles` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:53.871545 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf35c35-7186-4022-8c95-04caa75bad8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:53.871925 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1052/2102] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:23:53 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:23:53.874411 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf35c35-7186-4022-8c95-04caa75bad8f admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:53.908310 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1051/2103] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:23:53 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:23:53.913922 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f535f85b-54cc-40ae-b1c9-3f3a055359ff service cinder] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:53.925038 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f535f85b-54cc-40ae-b1c9-3f3a055359ff service cinder] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:53.930719 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef37db9-67cd-4687-a182-d38609fd368d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:53.931473 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef37db9-67cd-4687-a182-d38609fd368d admin admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:53.931907 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef37db9-67cd-4687-a182-d38609fd368d admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:53.932389 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef37db9-67cd-4687-a182-d38609fd368d admin admin] PATH_INFO: `/v3/roles` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:53.933152 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef37db9-67cd-4687-a182-d38609fd368d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:53.935002 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef37db9-67cd-4687-a182-d38609fd368d admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:53.946016 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1053/2104] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:23:53 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:23:53.950533 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf35c35-7186-4022-8c95-04caa75bad8f admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:53.962135 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf35c35-7186-4022-8c95-04caa75bad8f admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:53.967525 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1066d186-4edb-4826-a4be-4f0d4b4aa668 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:53.968350 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1066d186-4edb-4826-a4be-4f0d4b4aa668 admin admin] REQUEST_METHOD: `PUT` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:53.969084 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1066d186-4edb-4826-a4be-4f0d4b4aa668 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:53.969687 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1066d186-4edb-4826-a4be-4f0d4b4aa668 admin admin] PATH_INFO: `/v3/projects/cb4e4930b11847c6a9a0a448a7844e55/users/3cc7e32a4b644ecfa7efd482508f28f3/roles/d0b9768f4d014766b9326387f2e11f34` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:53.996132 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1066d186-4edb-4826-a4be-4f0d4b4aa668 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb4e4930b11847c6a9a0a448a7844e55, user_id=3cc7e32a4b644ecfa7efd482508f28f3, role_id=d0b9768f4d014766b9326387f2e11f34)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:53.998679 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1066d186-4edb-4826-a4be-4f0d4b4aa668 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:53.999878 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef37db9-67cd-4687-a182-d38609fd368d admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:54.010399 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef37db9-67cd-4687-a182-d38609fd368d admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:54.014218 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0948bf39-2d96-4862-ac2d-614483bafd6b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5c8fc69c518412ba1aad61a9f4f558c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:54.014894 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0948bf39-2d96-4862-ac2d-614483bafd6b service cinder] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:54.015251 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0948bf39-2d96-4862-ac2d-614483bafd6b service cinder] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:54.015599 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0948bf39-2d96-4862-ac2d-614483bafd6b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:54.017619 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-0948bf39-2d96-4862-ac2d-614483bafd6b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:23:54.076483 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1052/2105] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 01:23:53 2022] PUT /identity/v3/projects/cb4e4930b11847c6a9a0a448a7844e55/users/3cc7e32a4b644ecfa7efd482508f28f3/roles/d0b9768f4d014766b9326387f2e11f34 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:23:54.078648 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0948bf39-2d96-4862-ac2d-614483bafd6b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:54.080379 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0948bf39-2d96-4862-ac2d-614483bafd6b service cinder] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:54.088016 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7229d328-b000-4b95-a38f-239a52029188 None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:54.088308 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7229d328-b000-4b95-a38f-239a52029188 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:54.088609 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7229d328-b000-4b95-a38f-239a52029188 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:54.090075 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1054/2106] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:23:53 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 01:23:54.114313 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0948bf39-2d96-4862-ac2d-614483bafd6b service cinder] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:54.115986 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-0948bf39-2d96-4862-ac2d-614483bafd6b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:23:54.131993 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-7229d328-b000-4b95-a38f-239a52029188 None None] MFA Rules not processed for user `3cc7e32a4b644ecfa7efd482508f28f3`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:23:54.133352 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-7229d328-b000-4b95-a38f-239a52029188 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:23:54.150857 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0948bf39-2d96-4862-ac2d-614483bafd6b service cinder] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:54.208107 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1053/2107] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:23:54 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 01:23:54.215095 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087d7723-2907-42ad-bb99-92dbdf40098f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:54.215871 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087d7723-2907-42ad-bb99-92dbdf40098f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:54.216164 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087d7723-2907-42ad-bb99-92dbdf40098f admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:54.216439 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087d7723-2907-42ad-bb99-92dbdf40098f admin admin] PATH_INFO: `/v3/projects/2127f49c51164180ab4dfd834bab406f` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:54.217644 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087d7723-2907-42ad-bb99-92dbdf40098f admin admin] RBAC: Authorizing `identity:delete_project(project_id=2127f49c51164180ab4dfd834bab406f)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:54.219565 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087d7723-2907-42ad-bb99-92dbdf40098f admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:54.241198 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7229d328-b000-4b95-a38f-239a52029188 None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:54.259036 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:23:54.301098 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7229d328-b000-4b95-a38f-239a52029188 None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:54.304817 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51e0565-313f-42a6-8509-4d9b5a4446c3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5c8fc69c518412ba1aad61a9f4f558c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:54.305420 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51e0565-313f-42a6-8509-4d9b5a4446c3 service cinder] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:54.307972 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51e0565-313f-42a6-8509-4d9b5a4446c3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:54.307972 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51e0565-313f-42a6-8509-4d9b5a4446c3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:54.313972 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51e0565-313f-42a6-8509-4d9b5a4446c3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:54.315457 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51e0565-313f-42a6-8509-4d9b5a4446c3 service cinder] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:54.324426 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1054/2108] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:23:54 2022] GET /identity/v3/auth/tokens => generated 3272 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:23:54.346193 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-087d7723-2907-42ad-bb99-92dbdf40098f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2127f49c51164180ab4dfd834bab406f', 'request_id': 'req-087d7723-2907-42ad-bb99-92dbdf40098f'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:54.353371 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-087d7723-2907-42ad-bb99-92dbdf40098f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2127f49c51164180ab4dfd834bab406f', 'request_id': 'req-087d7723-2907-42ad-bb99-92dbdf40098f'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:54.459423 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-087d7723-2907-42ad-bb99-92dbdf40098f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2127f49c51164180ab4dfd834bab406f', 'request_id': 'req-087d7723-2907-42ad-bb99-92dbdf40098f'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:54.475269 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-087d7723-2907-42ad-bb99-92dbdf40098f admin admin] The token cache is being invalidate because project 2127f49c51164180ab4dfd834bab406f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81750) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 01:23:54.475715 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-087d7723-2907-42ad-bb99-92dbdf40098f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:54.476665 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-087d7723-2907-42ad-bb99-92dbdf40098f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:23:54.479111 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1055/2109] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 01:23:54 2022] DELETE /identity/v3/projects/2127f49c51164180ab4dfd834bab406f => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:23:54.483940 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee02e2b4-b102-4593-8f75-43f7d1d9f491 None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:54.484231 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee02e2b4-b102-4593-8f75-43f7d1d9f491 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:54.484498 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee02e2b4-b102-4593-8f75-43f7d1d9f491 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:54.510563 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-ee02e2b4-b102-4593-8f75-43f7d1d9f491 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:23:54.512194 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-ee02e2b4-b102-4593-8f75-43f7d1d9f491 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:23:54.605902 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1055/2110] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:23:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:23:54.615012 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575e6d72-5420-4d85-9c8b-cb7dd0ca937b None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:54.615302 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575e6d72-5420-4d85-9c8b-cb7dd0ca937b None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:54.615758 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575e6d72-5420-4d85-9c8b-cb7dd0ca937b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:54.643556 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-575e6d72-5420-4d85-9c8b-cb7dd0ca937b None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:23:54.645206 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-575e6d72-5420-4d85-9c8b-cb7dd0ca937b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:23:54.657771 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1056/2111] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:23:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:23:54.663749 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee02e2b4-b102-4593-8f75-43f7d1d9f491 None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:54.674167 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee02e2b4-b102-4593-8f75-43f7d1d9f491 None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:54.681946 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfaeccc-5a97-42f2-aa94-3cb21d5b6f6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:54.682672 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfaeccc-5a97-42f2-aa94-3cb21d5b6f6c admin admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:54.683850 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfaeccc-5a97-42f2-aa94-3cb21d5b6f6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:54.684388 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfaeccc-5a97-42f2-aa94-3cb21d5b6f6c admin admin] PATH_INFO: `/v3/domains` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:54.685510 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfaeccc-5a97-42f2-aa94-3cb21d5b6f6c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:23:54.685876 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfaeccc-5a97-42f2-aa94-3cb21d5b6f6c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:54.688006 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfaeccc-5a97-42f2-aa94-3cb21d5b6f6c admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:54.701759 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1056/2112] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Aug 31 01:23:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:23:54.706186 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575e6d72-5420-4d85-9c8b-cb7dd0ca937b None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:54.717670 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575e6d72-5420-4d85-9c8b-cb7dd0ca937b None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:54.723453 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977c18aa-c5ea-4532-92dc-62872d9a8221 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:54.724312 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977c18aa-c5ea-4532-92dc-62872d9a8221 admin admin] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:54.724818 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977c18aa-c5ea-4532-92dc-62872d9a8221 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:54.725282 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977c18aa-c5ea-4532-92dc-62872d9a8221 admin admin] PATH_INFO: `/v3/projects` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:54.726117 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977c18aa-c5ea-4532-92dc-62872d9a8221 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:54.728233 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977c18aa-c5ea-4532-92dc-62872d9a8221 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:54.803025 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:23:54.853234 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1057/2113] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 01:23:54 2022] POST /identity/v3/projects => generated 391 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:23:54.857374 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfaeccc-5a97-42f2-aa94-3cb21d5b6f6c admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:54.880750 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfaeccc-5a97-42f2-aa94-3cb21d5b6f6c admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:54.962745 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501194e5-9876-4a92-a620-6db1f65914ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:54.962745 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501194e5-9876-4a92-a620-6db1f65914ef admin admin] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:54.962745 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501194e5-9876-4a92-a620-6db1f65914ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:54.962745 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501194e5-9876-4a92-a620-6db1f65914ef admin admin] PATH_INFO: `/v3/users` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:54.963066 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501194e5-9876-4a92-a620-6db1f65914ef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:54.964791 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501194e5-9876-4a92-a620-6db1f65914ef admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:55.071986 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1057/2114] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 01:23:54 2022] POST /identity/v3/users => generated 296 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 01:23:55.073794 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977c18aa-c5ea-4532-92dc-62872d9a8221 admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:55.088596 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977c18aa-c5ea-4532-92dc-62872d9a8221 admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:55.094532 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2234df83-4f8a-45df-9b72-ae7de50254ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:55.096319 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2234df83-4f8a-45df-9b72-ae7de50254ec admin admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:55.096600 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2234df83-4f8a-45df-9b72-ae7de50254ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:55.096921 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2234df83-4f8a-45df-9b72-ae7de50254ec admin admin] PATH_INFO: `/v3/roles` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:55.102225 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2234df83-4f8a-45df-9b72-ae7de50254ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:55.102225 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2234df83-4f8a-45df-9b72-ae7de50254ec admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:55.114259 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1058/2115] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:23:55 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 01:23:55.115727 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501194e5-9876-4a92-a620-6db1f65914ef admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:55.130395 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501194e5-9876-4a92-a620-6db1f65914ef admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:55.136478 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e577c29-f5bb-48e1-862a-e1d31dc40a94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:55.137213 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e577c29-f5bb-48e1-862a-e1d31dc40a94 admin admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:55.137741 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e577c29-f5bb-48e1-862a-e1d31dc40a94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:55.138002 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e577c29-f5bb-48e1-862a-e1d31dc40a94 admin admin] PATH_INFO: `/v3/roles` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:55.138889 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e577c29-f5bb-48e1-862a-e1d31dc40a94 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:55.140545 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e577c29-f5bb-48e1-862a-e1d31dc40a94 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:55.154834 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1058/2116] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:23:55 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:23:55.160500 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2234df83-4f8a-45df-9b72-ae7de50254ec admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:55.197198 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2234df83-4f8a-45df-9b72-ae7de50254ec admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:55.205163 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db86a451-c864-4a91-8dc8-c62a9d325b77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:55.206210 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db86a451-c864-4a91-8dc8-c62a9d325b77 admin admin] REQUEST_METHOD: `PUT` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:55.206916 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db86a451-c864-4a91-8dc8-c62a9d325b77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:55.207380 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db86a451-c864-4a91-8dc8-c62a9d325b77 admin admin] PATH_INFO: `/v3/projects/3d340ee4f1f74140aa321c30ed1b9456/users/0833316657274dda842a118a79fba788/roles/d0b9768f4d014766b9326387f2e11f34` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:55.237140 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db86a451-c864-4a91-8dc8-c62a9d325b77 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d340ee4f1f74140aa321c30ed1b9456, user_id=0833316657274dda842a118a79fba788, role_id=d0b9768f4d014766b9326387f2e11f34)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:55.240184 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db86a451-c864-4a91-8dc8-c62a9d325b77 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:55.324126 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ea5838-c0bc-43a8-b0fd-261caa313b10 None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:55.324398 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ea5838-c0bc-43a8-b0fd-261caa313b10 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:55.324674 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ea5838-c0bc-43a8-b0fd-261caa313b10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:55.328499 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1059/2117] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 01:23:55 2022] PUT /identity/v3/projects/3d340ee4f1f74140aa321c30ed1b9456/users/0833316657274dda842a118a79fba788/roles/d0b9768f4d014766b9326387f2e11f34 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:23:55.358594 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-91ea5838-c0bc-43a8-b0fd-261caa313b10 None None] MFA Rules not processed for user `0833316657274dda842a118a79fba788`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:23:55.359828 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-91ea5838-c0bc-43a8-b0fd-261caa313b10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:23:55.368286 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:23:55.430250 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1059/2118] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:23:55 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:23:55.442156 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db86a451-c864-4a91-8dc8-c62a9d325b77 admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:23:55.443875 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-db86a451-c864-4a91-8dc8-c62a9d325b77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:23:55.502412 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db86a451-c864-4a91-8dc8-c62a9d325b77 admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:23:55.506187 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb230377-1b0a-4cba-8fc5-8c233a5d320e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5c8fc69c518412ba1aad61a9f4f558c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:23:55.506804 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb230377-1b0a-4cba-8fc5-8c233a5d320e service cinder] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:23:55.507088 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb230377-1b0a-4cba-8fc5-8c233a5d320e service cinder] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:23:55.507365 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb230377-1b0a-4cba-8fc5-8c233a5d320e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:23:55.517346 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb230377-1b0a-4cba-8fc5-8c233a5d320e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:23:55.518816 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb230377-1b0a-4cba-8fc5-8c233a5d320e service cinder] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:23:55.529383 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1060/2119] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:23:55 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:23:57.531940 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:23:57.710894 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:DELETE=9,keystone:SELECT=89,keystone:INSERT=2 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:23:59.088355 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:23:59.122357 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:23:59.491994 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:23:59.520938 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=95,keystone:DELETE=1,keystone:INSERT=4 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:23:59.796711 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:23:59.846348 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:23:59.932047 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:24:00.013540 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:24:00.070342 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:24:00.104584 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:24:00.139250 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:24:05.106576 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ea5838-c0bc-43a8-b0fd-261caa313b10 None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:24:05.107645 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-91ea5838-c0bc-43a8-b0fd-261caa313b10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:24:05.161696 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:24:05.174235 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ea5838-c0bc-43a8-b0fd-261caa313b10 None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:24:05.179591 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a86628-5856-4159-b2f0-c5d87808fa18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:24:05.180219 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a86628-5856-4159-b2f0-c5d87808fa18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:24:05.180503 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a86628-5856-4159-b2f0-c5d87808fa18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:24:05.180815 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a86628-5856-4159-b2f0-c5d87808fa18 admin admin] PATH_INFO: `/v3/users/9755cee910ff49a6ad059a9cb02989e6` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:24:05.182009 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a86628-5856-4159-b2f0-c5d87808fa18 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9755cee910ff49a6ad059a9cb02989e6)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:24:05.184036 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a86628-5856-4159-b2f0-c5d87808fa18 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:24:05.403660 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-57a86628-5856-4159-b2f0-c5d87808fa18 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9755cee910ff49a6ad059a9cb02989e6'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:24:05.491255 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-57a86628-5856-4159-b2f0-c5d87808fa18 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9755cee910ff49a6ad059a9cb02989e6'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:24:05.491885 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-57a86628-5856-4159-b2f0-c5d87808fa18 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9755cee910ff49a6ad059a9cb02989e6'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:24:05.499734 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-57a86628-5856-4159-b2f0-c5d87808fa18 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9755cee910ff49a6ad059a9cb02989e6'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:24:05.500406 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-57a86628-5856-4159-b2f0-c5d87808fa18 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9755cee910ff49a6ad059a9cb02989e6'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:24:05.518434 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1060/2120] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 01:24:05 2022] DELETE /identity/v3/users/9755cee910ff49a6ad059a9cb02989e6 => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:24:05.522999 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f391c982-7d5c-460b-97d6-896deade8e2a None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:24:05.523441 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f391c982-7d5c-460b-97d6-896deade8e2a None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:24:05.523903 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f391c982-7d5c-460b-97d6-896deade8e2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:24:05.540443 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:24:05.546756 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-f391c982-7d5c-460b-97d6-896deade8e2a None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:24:05.548184 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-f391c982-7d5c-460b-97d6-896deade8e2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:24:05.626666 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1061/2121] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:24:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:24:05.633580 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a86628-5856-4159-b2f0-c5d87808fa18 admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:24:05.635134 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-57a86628-5856-4159-b2f0-c5d87808fa18 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:24:05.699300 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a86628-5856-4159-b2f0-c5d87808fa18 admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:24:05.703056 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27067f19-be85-4f55-9a9a-6ed4cfacd183 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0817c3320d5d41d8bba0550c8ae8dd60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:24:05.706762 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27067f19-be85-4f55-9a9a-6ed4cfacd183 service neutron] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:24:05.706762 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27067f19-be85-4f55-9a9a-6ed4cfacd183 service neutron] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:24:05.706762 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27067f19-be85-4f55-9a9a-6ed4cfacd183 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:24:05.714580 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27067f19-be85-4f55-9a9a-6ed4cfacd183 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:24:05.716015 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27067f19-be85-4f55-9a9a-6ed4cfacd183 service neutron] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:24:05.726295 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1061/2122] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:24:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:24:05.974225 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f391c982-7d5c-460b-97d6-896deade8e2a None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:24:05.975654 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-f391c982-7d5c-460b-97d6-896deade8e2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:24:05.987776 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f391c982-7d5c-460b-97d6-896deade8e2a None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:24:05.994734 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35eb6624-376b-403a-a467-16f4c5250998 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:24:05.995381 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35eb6624-376b-403a-a467-16f4c5250998 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:24:05.995663 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35eb6624-376b-403a-a467-16f4c5250998 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:24:05.995937 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35eb6624-376b-403a-a467-16f4c5250998 admin admin] PATH_INFO: `/v3/users/3cc7e32a4b644ecfa7efd482508f28f3` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:24:05.997557 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35eb6624-376b-403a-a467-16f4c5250998 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3cc7e32a4b644ecfa7efd482508f28f3)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:24:05.999083 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35eb6624-376b-403a-a467-16f4c5250998 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:24:06.163479 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27067f19-be85-4f55-9a9a-6ed4cfacd183 service neutron] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:24:06.165053 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-27067f19-be85-4f55-9a9a-6ed4cfacd183 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:24:06.177956 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27067f19-be85-4f55-9a9a-6ed4cfacd183 service neutron] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:24:06.182555 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ba7f08-f566-42ff-ad70-a99061655623 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:24:06.183191 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ba7f08-f566-42ff-ad70-a99061655623 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:24:06.183474 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ba7f08-f566-42ff-ad70-a99061655623 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:24:06.183755 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ba7f08-f566-42ff-ad70-a99061655623 admin admin] PATH_INFO: `/v3/projects/cd203cd3bc9f472ea6e7553ee837c677` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:24:06.184955 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ba7f08-f566-42ff-ad70-a99061655623 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cd203cd3bc9f472ea6e7553ee837c677)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:24:06.186849 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ba7f08-f566-42ff-ad70-a99061655623 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:24:06.206681 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-35eb6624-376b-403a-a467-16f4c5250998 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3cc7e32a4b644ecfa7efd482508f28f3'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:24:06.236037 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-35eb6624-376b-403a-a467-16f4c5250998 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3cc7e32a4b644ecfa7efd482508f28f3'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:24:06.236896 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-35eb6624-376b-403a-a467-16f4c5250998 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3cc7e32a4b644ecfa7efd482508f28f3'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:24:06.244914 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-35eb6624-376b-403a-a467-16f4c5250998 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3cc7e32a4b644ecfa7efd482508f28f3'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:24:06.293201 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-35eb6624-376b-403a-a467-16f4c5250998 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3cc7e32a4b644ecfa7efd482508f28f3'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:24:06.297597 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-19ba7f08-f566-42ff-ad70-a99061655623 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cd203cd3bc9f472ea6e7553ee837c677', 'request_id': 'req-19ba7f08-f566-42ff-ad70-a99061655623'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:24:06.300853 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1062/2123] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 01:24:05 2022] DELETE /identity/v3/users/3cc7e32a4b644ecfa7efd482508f28f3 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:24:06.305842 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05eacf74-8407-4a18-afa8-766ec9af02b7 None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:24:06.306271 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05eacf74-8407-4a18-afa8-766ec9af02b7 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:24:06.306672 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05eacf74-8407-4a18-afa8-766ec9af02b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:24:06.310397 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-19ba7f08-f566-42ff-ad70-a99061655623 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cd203cd3bc9f472ea6e7553ee837c677', 'request_id': 'req-19ba7f08-f566-42ff-ad70-a99061655623'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:24:06.319805 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-19ba7f08-f566-42ff-ad70-a99061655623 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cd203cd3bc9f472ea6e7553ee837c677', 'request_id': 'req-19ba7f08-f566-42ff-ad70-a99061655623'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:24:06.328430 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-05eacf74-8407-4a18-afa8-766ec9af02b7 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:24:06.330013 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-05eacf74-8407-4a18-afa8-766ec9af02b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:24:06.398393 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1063/2124] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:24:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:24:06.408566 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05eacf74-8407-4a18-afa8-766ec9af02b7 None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:24:06.410025 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-05eacf74-8407-4a18-afa8-766ec9af02b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:24:06.419652 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:24:06.424753 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-19ba7f08-f566-42ff-ad70-a99061655623 admin admin] The token cache is being invalidate because project cd203cd3bc9f472ea6e7553ee837c677 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81749) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 01:24:06.427621 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-19ba7f08-f566-42ff-ad70-a99061655623 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:24:06.427621 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-19ba7f08-f566-42ff-ad70-a99061655623 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:24:06.427940 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1062/2125] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 01:24:06 2022] DELETE /identity/v3/projects/cd203cd3bc9f472ea6e7553ee837c677 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:24:06.431644 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ba7f08-f566-42ff-ad70-a99061655623 admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:24:06.432660 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:24:06.433038 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-19ba7f08-f566-42ff-ad70-a99061655623 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:24:06.468937 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05eacf74-8407-4a18-afa8-766ec9af02b7 None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:24:06.472633 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d10e1fc-aca5-4487-97be-3f2fade0fc1c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0817c3320d5d41d8bba0550c8ae8dd60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:24:06.474258 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d10e1fc-aca5-4487-97be-3f2fade0fc1c service neutron] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:24:06.474258 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d10e1fc-aca5-4487-97be-3f2fade0fc1c service neutron] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:24:06.474258 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d10e1fc-aca5-4487-97be-3f2fade0fc1c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:24:06.475812 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-4d10e1fc-aca5-4487-97be-3f2fade0fc1c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:24:06.488299 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d10e1fc-aca5-4487-97be-3f2fade0fc1c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:24:06.490248 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d10e1fc-aca5-4487-97be-3f2fade0fc1c service neutron] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:24:06.497734 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ba7f08-f566-42ff-ad70-a99061655623 admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:24:06.499802 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1064/2126] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:24:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 01:24:06.502417 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df8b64e-38ef-446d-a80b-2ed0847266b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:24:06.503070 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df8b64e-38ef-446d-a80b-2ed0847266b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:24:06.503356 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df8b64e-38ef-446d-a80b-2ed0847266b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:24:06.503635 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df8b64e-38ef-446d-a80b-2ed0847266b6 admin admin] PATH_INFO: `/v3/users/4e18f7aded084c76a4580dfaea91d397` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:24:06.505453 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df8b64e-38ef-446d-a80b-2ed0847266b6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4e18f7aded084c76a4580dfaea91d397)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:24:06.507410 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df8b64e-38ef-446d-a80b-2ed0847266b6 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:24:06.713791 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-5df8b64e-38ef-446d-a80b-2ed0847266b6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4e18f7aded084c76a4580dfaea91d397'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:24:06.789293 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-5df8b64e-38ef-446d-a80b-2ed0847266b6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4e18f7aded084c76a4580dfaea91d397'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:24:06.789920 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-5df8b64e-38ef-446d-a80b-2ed0847266b6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4e18f7aded084c76a4580dfaea91d397'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:24:06.797807 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-5df8b64e-38ef-446d-a80b-2ed0847266b6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4e18f7aded084c76a4580dfaea91d397'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:24:06.798602 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-5df8b64e-38ef-446d-a80b-2ed0847266b6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4e18f7aded084c76a4580dfaea91d397'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:24:06.817878 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1063/2127] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 01:24:06 2022] DELETE /identity/v3/users/4e18f7aded084c76a4580dfaea91d397 => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 01:24:07.143611 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d10e1fc-aca5-4487-97be-3f2fade0fc1c service neutron] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:24:07.145983 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-4d10e1fc-aca5-4487-97be-3f2fade0fc1c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:24:07.159340 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d10e1fc-aca5-4487-97be-3f2fade0fc1c service neutron] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:24:07.224452 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ea0435-1c2b-4bf9-8a0a-73f06e0c7573 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:24:07.225132 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ea0435-1c2b-4bf9-8a0a-73f06e0c7573 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:24:07.225417 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ea0435-1c2b-4bf9-8a0a-73f06e0c7573 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:24:07.225692 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ea0435-1c2b-4bf9-8a0a-73f06e0c7573 admin admin] PATH_INFO: `/v3/projects/cb4e4930b11847c6a9a0a448a7844e55` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:24:07.226820 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ea0435-1c2b-4bf9-8a0a-73f06e0c7573 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb4e4930b11847c6a9a0a448a7844e55)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:24:07.228804 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ea0435-1c2b-4bf9-8a0a-73f06e0c7573 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:24:07.360333 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-e7ea0435-1c2b-4bf9-8a0a-73f06e0c7573 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb4e4930b11847c6a9a0a448a7844e55', 'request_id': 'req-e7ea0435-1c2b-4bf9-8a0a-73f06e0c7573'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:24:07.368208 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-e7ea0435-1c2b-4bf9-8a0a-73f06e0c7573 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb4e4930b11847c6a9a0a448a7844e55', 'request_id': 'req-e7ea0435-1c2b-4bf9-8a0a-73f06e0c7573'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:24:07.463957 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-e7ea0435-1c2b-4bf9-8a0a-73f06e0c7573 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb4e4930b11847c6a9a0a448a7844e55', 'request_id': 'req-e7ea0435-1c2b-4bf9-8a0a-73f06e0c7573'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:24:07.475487 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-e7ea0435-1c2b-4bf9-8a0a-73f06e0c7573 admin admin] The token cache is being invalidate because project cb4e4930b11847c6a9a0a448a7844e55 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81750) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 01:24:07.475764 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-e7ea0435-1c2b-4bf9-8a0a-73f06e0c7573 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:24:07.476383 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-e7ea0435-1c2b-4bf9-8a0a-73f06e0c7573 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:24:07.477888 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1065/2128] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 01:24:07 2022] DELETE /identity/v3/projects/cb4e4930b11847c6a9a0a448a7844e55 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:24:07.516242 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df8b64e-38ef-446d-a80b-2ed0847266b6 admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:24:07.517849 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-5df8b64e-38ef-446d-a80b-2ed0847266b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:24:07.534597 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df8b64e-38ef-446d-a80b-2ed0847266b6 admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:24:07.601030 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fecc00d-4ad0-4b9a-91ca-327eaee329ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:24:07.601696 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fecc00d-4ad0-4b9a-91ca-327eaee329ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:24:07.601980 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fecc00d-4ad0-4b9a-91ca-327eaee329ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:24:07.602259 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fecc00d-4ad0-4b9a-91ca-327eaee329ea admin admin] PATH_INFO: `/v3/projects/69c2c084da1c493d9e4747639e68739f` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:24:07.603439 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fecc00d-4ad0-4b9a-91ca-327eaee329ea admin admin] RBAC: Authorizing `identity:delete_project(project_id=69c2c084da1c493d9e4747639e68739f)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:24:07.605545 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fecc00d-4ad0-4b9a-91ca-327eaee329ea admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:24:07.629058 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:24:07.761814 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-3fecc00d-4ad0-4b9a-91ca-327eaee329ea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '69c2c084da1c493d9e4747639e68739f', 'request_id': 'req-3fecc00d-4ad0-4b9a-91ca-327eaee329ea'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:24:07.767250 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-3fecc00d-4ad0-4b9a-91ca-327eaee329ea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '69c2c084da1c493d9e4747639e68739f', 'request_id': 'req-3fecc00d-4ad0-4b9a-91ca-327eaee329ea'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:24:07.773424 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-3fecc00d-4ad0-4b9a-91ca-327eaee329ea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '69c2c084da1c493d9e4747639e68739f', 'request_id': 'req-3fecc00d-4ad0-4b9a-91ca-327eaee329ea'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:24:07.857780 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-3fecc00d-4ad0-4b9a-91ca-327eaee329ea admin admin] The token cache is being invalidate because project 69c2c084da1c493d9e4747639e68739f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81749) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 01:24:07.858080 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-3fecc00d-4ad0-4b9a-91ca-327eaee329ea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:24:07.858707 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-3fecc00d-4ad0-4b9a-91ca-327eaee329ea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:24:07.860195 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1064/2129] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 01:24:07 2022] DELETE /identity/v3/projects/69c2c084da1c493d9e4747639e68739f => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:24:07.864837 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ea0435-1c2b-4bf9-8a0a-73f06e0c7573 admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:24:07.866251 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-e7ea0435-1c2b-4bf9-8a0a-73f06e0c7573 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:24:07.878731 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ea0435-1c2b-4bf9-8a0a-73f06e0c7573 admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:24:07.928471 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:24:07.937207 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad361649-46e0-45cc-925f-eece79bf049f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:24:07.942555 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad361649-46e0-45cc-925f-eece79bf049f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:24:07.942555 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad361649-46e0-45cc-925f-eece79bf049f admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:24:07.942555 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad361649-46e0-45cc-925f-eece79bf049f admin admin] PATH_INFO: `/v3/users/58ec6f35f3d94f03846a171a050f13a6` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:24:07.942555 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad361649-46e0-45cc-925f-eece79bf049f admin admin] RBAC: Authorizing `identity:delete_user(user_id=58ec6f35f3d94f03846a171a050f13a6)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:24:07.942555 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad361649-46e0-45cc-925f-eece79bf049f admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:24:08.152975 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-ad361649-46e0-45cc-925f-eece79bf049f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '58ec6f35f3d94f03846a171a050f13a6'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:24:08.166021 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-ad361649-46e0-45cc-925f-eece79bf049f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '58ec6f35f3d94f03846a171a050f13a6'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:24:08.166750 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-ad361649-46e0-45cc-925f-eece79bf049f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '58ec6f35f3d94f03846a171a050f13a6'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:24:08.173579 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-ad361649-46e0-45cc-925f-eece79bf049f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '58ec6f35f3d94f03846a171a050f13a6'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:24:08.234174 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-ad361649-46e0-45cc-925f-eece79bf049f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '58ec6f35f3d94f03846a171a050f13a6'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:24:08.241225 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1066/2130] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 01:24:07 2022] DELETE /identity/v3/users/58ec6f35f3d94f03846a171a050f13a6 => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:24:08.717098 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fecc00d-4ad0-4b9a-91ca-327eaee329ea admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:24:08.719324 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-3fecc00d-4ad0-4b9a-91ca-327eaee329ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:24:08.733561 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fecc00d-4ad0-4b9a-91ca-327eaee329ea admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:24:08.792984 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b54a9a-0a79-48e7-bace-f0b0837beaf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:24:08.793627 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b54a9a-0a79-48e7-bace-f0b0837beaf5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:24:08.793911 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b54a9a-0a79-48e7-bace-f0b0837beaf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:24:08.794187 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b54a9a-0a79-48e7-bace-f0b0837beaf5 admin admin] PATH_INFO: `/v3/projects/7d4a7299781446f998ddeb413556d2b6` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:24:08.795328 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b54a9a-0a79-48e7-bace-f0b0837beaf5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7d4a7299781446f998ddeb413556d2b6)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:24:08.797297 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b54a9a-0a79-48e7-bace-f0b0837beaf5 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:24:08.939332 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-62b54a9a-0a79-48e7-bace-f0b0837beaf5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7d4a7299781446f998ddeb413556d2b6', 'request_id': 'req-62b54a9a-0a79-48e7-bace-f0b0837beaf5'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:24:08.945386 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-62b54a9a-0a79-48e7-bace-f0b0837beaf5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7d4a7299781446f998ddeb413556d2b6', 'request_id': 'req-62b54a9a-0a79-48e7-bace-f0b0837beaf5'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:24:08.951651 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-62b54a9a-0a79-48e7-bace-f0b0837beaf5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7d4a7299781446f998ddeb413556d2b6', 'request_id': 'req-62b54a9a-0a79-48e7-bace-f0b0837beaf5'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:24:09.047642 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-62b54a9a-0a79-48e7-bace-f0b0837beaf5 admin admin] The token cache is being invalidate because project 7d4a7299781446f998ddeb413556d2b6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81749) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 01:24:09.047867 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-62b54a9a-0a79-48e7-bace-f0b0837beaf5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:24:09.048461 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-62b54a9a-0a79-48e7-bace-f0b0837beaf5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:24:09.049999 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1065/2131] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 01:24:08 2022] DELETE /identity/v3/projects/7d4a7299781446f998ddeb413556d2b6 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:24:09.056029 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ea627b-8af6-46cb-aa39-b36852a0f107 None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:24:09.056321 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ea627b-8af6-46cb-aa39-b36852a0f107 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:24:09.056599 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ea627b-8af6-46cb-aa39-b36852a0f107 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:24:09.076985 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-f0ea627b-8af6-46cb-aa39-b36852a0f107 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:24:09.078339 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-f0ea627b-8af6-46cb-aa39-b36852a0f107 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:24:09.147276 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1067/2132] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:24:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:24:09.155718 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35192d7b-2950-4968-b972-e351edb1bc82 None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:24:09.156130 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35192d7b-2950-4968-b972-e351edb1bc82 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:24:09.156409 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35192d7b-2950-4968-b972-e351edb1bc82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:24:09.178790 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-35192d7b-2950-4968-b972-e351edb1bc82 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:24:09.180104 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-35192d7b-2950-4968-b972-e351edb1bc82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:24:09.189040 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1066/2133] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:24:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 01:24:09.193761 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ea627b-8af6-46cb-aa39-b36852a0f107 None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:24:09.205067 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ea627b-8af6-46cb-aa39-b36852a0f107 None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:24:09.210776 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35626022-b781-4db9-9732-50512e6b935d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:24:09.211460 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35626022-b781-4db9-9732-50512e6b935d admin admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:24:09.217787 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35626022-b781-4db9-9732-50512e6b935d admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:24:09.217787 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35626022-b781-4db9-9732-50512e6b935d admin admin] PATH_INFO: `/v3/domains` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:24:09.217787 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35626022-b781-4db9-9732-50512e6b935d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:24:09.217787 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35626022-b781-4db9-9732-50512e6b935d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:24:09.217787 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35626022-b781-4db9-9732-50512e6b935d admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:24:09.231249 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1068/2134] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Aug 31 01:24:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:24:09.235800 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35192d7b-2950-4968-b972-e351edb1bc82 None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:24:09.245860 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35192d7b-2950-4968-b972-e351edb1bc82 None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:24:09.250727 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548997b1-ee13-4920-8117-e841e46c97b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:24:09.251338 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548997b1-ee13-4920-8117-e841e46c97b7 admin admin] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:24:09.251779 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548997b1-ee13-4920-8117-e841e46c97b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:24:09.252066 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548997b1-ee13-4920-8117-e841e46c97b7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:24:09.252851 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548997b1-ee13-4920-8117-e841e46c97b7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:24:09.254828 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548997b1-ee13-4920-8117-e841e46c97b7 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:24:09.361417 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1067/2135] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 01:24:09 2022] POST /identity/v3/projects => generated 379 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:24:09.362584 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35626022-b781-4db9-9732-50512e6b935d admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:24:09.374903 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35626022-b781-4db9-9732-50512e6b935d admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:24:09.460043 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8221426-caa7-4eee-8fb2-466d9dfb04ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:24:09.466468 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8221426-caa7-4eee-8fb2-466d9dfb04ea admin admin] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:24:09.466973 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8221426-caa7-4eee-8fb2-466d9dfb04ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:24:09.467253 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8221426-caa7-4eee-8fb2-466d9dfb04ea admin admin] PATH_INFO: `/v3/users` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:24:09.467920 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8221426-caa7-4eee-8fb2-466d9dfb04ea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:24:09.469900 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8221426-caa7-4eee-8fb2-466d9dfb04ea admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:24:09.478407 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:24:09.596311 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1069/2136] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 01:24:09 2022] POST /identity/v3/users => generated 290 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:24:09.601221 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548997b1-ee13-4920-8117-e841e46c97b7 admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:24:09.611852 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548997b1-ee13-4920-8117-e841e46c97b7 admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:24:09.617567 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19f6825-5d4a-483a-8ac9-592043c89065 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:24:09.618199 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19f6825-5d4a-483a-8ac9-592043c89065 admin admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:24:09.618485 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19f6825-5d4a-483a-8ac9-592043c89065 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:24:09.622557 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19f6825-5d4a-483a-8ac9-592043c89065 admin admin] PATH_INFO: `/v3/roles` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:24:09.622557 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19f6825-5d4a-483a-8ac9-592043c89065 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:24:09.622557 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19f6825-5d4a-483a-8ac9-592043c89065 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:24:09.623574 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:24:09.631950 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1068/2137] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:24:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:24:09.636022 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8221426-caa7-4eee-8fb2-466d9dfb04ea admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:24:09.646088 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8221426-caa7-4eee-8fb2-466d9dfb04ea admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:24:09.650665 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa466ef0-2eb4-43f8-a9cf-fe1317f76d16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:24:09.651251 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa466ef0-2eb4-43f8-a9cf-fe1317f76d16 admin admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:24:09.651530 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa466ef0-2eb4-43f8-a9cf-fe1317f76d16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:24:09.651804 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa466ef0-2eb4-43f8-a9cf-fe1317f76d16 admin admin] PATH_INFO: `/v3/roles` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:24:09.652435 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa466ef0-2eb4-43f8-a9cf-fe1317f76d16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:24:09.654048 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa466ef0-2eb4-43f8-a9cf-fe1317f76d16 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:24:09.664897 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1070/2138] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:24:09 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:24:09.668907 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19f6825-5d4a-483a-8ac9-592043c89065 admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:24:09.679391 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19f6825-5d4a-483a-8ac9-592043c89065 admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:24:09.683970 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20242bf-8a91-431a-8a7c-eea2130112b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:24:09.685721 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20242bf-8a91-431a-8a7c-eea2130112b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:24:09.685721 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20242bf-8a91-431a-8a7c-eea2130112b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:24:09.685721 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20242bf-8a91-431a-8a7c-eea2130112b4 admin admin] PATH_INFO: `/v3/projects/e95043f03e76408ea9613c61faf2b8b5/users/7865742f858940a78f838470daafe6e1/roles/d0b9768f4d014766b9326387f2e11f34` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:24:09.699667 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20242bf-8a91-431a-8a7c-eea2130112b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e95043f03e76408ea9613c61faf2b8b5, user_id=7865742f858940a78f838470daafe6e1, role_id=d0b9768f4d014766b9326387f2e11f34)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:24:09.702797 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20242bf-8a91-431a-8a7c-eea2130112b4 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:24:09.807884 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1069/2139] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 01:24:09 2022] PUT /identity/v3/projects/e95043f03e76408ea9613c61faf2b8b5/users/7865742f858940a78f838470daafe6e1/roles/d0b9768f4d014766b9326387f2e11f34 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:24:09.819218 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55d370d-e534-481a-ad9a-c91f662176b0 None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:24:09.819515 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55d370d-e534-481a-ad9a-c91f662176b0 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:24:09.819790 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55d370d-e534-481a-ad9a-c91f662176b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:24:09.847633 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-b55d370d-e534-481a-ad9a-c91f662176b0 None None] MFA Rules not processed for user `7865742f858940a78f838470daafe6e1`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:24:09.849575 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-b55d370d-e534-481a-ad9a-c91f662176b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:24:09.923544 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1071/2140] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:24:09 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:24:09.929068 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20242bf-8a91-431a-8a7c-eea2130112b4 admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:24:09.993468 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20242bf-8a91-431a-8a7c-eea2130112b4 admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:24:09.998388 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3c4bed-672e-430b-bec5-e4d532e4b581 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:24:09.999038 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3c4bed-672e-430b-bec5-e4d532e4b581 admin admin] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:24:09.999318 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3c4bed-672e-430b-bec5-e4d532e4b581 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:24:09.999592 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3c4bed-672e-430b-bec5-e4d532e4b581 admin admin] PATH_INFO: `/v3/projects` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:24:10.000213 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3c4bed-672e-430b-bec5-e4d532e4b581 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:24:10.002149 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3c4bed-672e-430b-bec5-e4d532e4b581 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:24:10.101005 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1070/2141] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 01:24:09 2022] POST /identity/v3/projects => generated 381 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 01:24:10.105180 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55d370d-e534-481a-ad9a-c91f662176b0 None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:24:10.116016 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55d370d-e534-481a-ad9a-c91f662176b0 None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:24:10.171568 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65614b7f-270d-461c-94ee-cdedd524dee4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:24:10.172198 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65614b7f-270d-461c-94ee-cdedd524dee4 admin admin] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:24:10.172463 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65614b7f-270d-461c-94ee-cdedd524dee4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:24:10.172759 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65614b7f-270d-461c-94ee-cdedd524dee4 admin admin] PATH_INFO: `/v3/users` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:24:10.173399 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65614b7f-270d-461c-94ee-cdedd524dee4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:24:10.175302 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65614b7f-270d-461c-94ee-cdedd524dee4 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:24:10.312087 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1072/2142] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 01:24:10 2022] POST /identity/v3/users => generated 297 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:24:10.313659 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3c4bed-672e-430b-bec5-e4d532e4b581 admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:24:10.324870 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3c4bed-672e-430b-bec5-e4d532e4b581 admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:24:10.329646 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018751b4-563a-4413-9b1c-90091c9644e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:24:10.330244 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018751b4-563a-4413-9b1c-90091c9644e6 admin admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:24:10.330497 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018751b4-563a-4413-9b1c-90091c9644e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:24:10.330768 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018751b4-563a-4413-9b1c-90091c9644e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:24:10.331405 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018751b4-563a-4413-9b1c-90091c9644e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:24:10.334023 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018751b4-563a-4413-9b1c-90091c9644e6 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:24:10.345223 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1071/2143] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:24:10 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:24:10.349608 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65614b7f-270d-461c-94ee-cdedd524dee4 admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:24:10.359783 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65614b7f-270d-461c-94ee-cdedd524dee4 admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:24:10.364784 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dde544-b0bf-42a2-9726-4fde8c7e7f6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:24:10.365446 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dde544-b0bf-42a2-9726-4fde8c7e7f6c admin admin] REQUEST_METHOD: `PUT` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:24:10.365736 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dde544-b0bf-42a2-9726-4fde8c7e7f6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:24:10.366017 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dde544-b0bf-42a2-9726-4fde8c7e7f6c admin admin] PATH_INFO: `/v3/projects/9fc17d26483749619d08476678422476/users/b24920dfe0344057a440c17f947bf56f/roles/d009411496b8452b88e77ffb8db015a7` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:24:10.382324 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27dde544-b0bf-42a2-9726-4fde8c7e7f6c admin admin] RBAC: Authorizing `identity:create_grant(project_id=9fc17d26483749619d08476678422476, user_id=b24920dfe0344057a440c17f947bf56f, role_id=d009411496b8452b88e77ffb8db015a7)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:24:10.385025 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27dde544-b0bf-42a2-9726-4fde8c7e7f6c admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:24:10.449639 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1073/2144] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 01:24:10 2022] PUT /identity/v3/projects/9fc17d26483749619d08476678422476/users/b24920dfe0344057a440c17f947bf56f/roles/d009411496b8452b88e77ffb8db015a7 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:24:10.454464 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018751b4-563a-4413-9b1c-90091c9644e6 admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:24:10.466967 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018751b4-563a-4413-9b1c-90091c9644e6 admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:24:10.531049 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d04367-1004-48ff-a9f7-38bd71a993e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:24:10.531668 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d04367-1004-48ff-a9f7-38bd71a993e7 admin admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:24:10.531955 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d04367-1004-48ff-a9f7-38bd71a993e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:24:10.532237 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d04367-1004-48ff-a9f7-38bd71a993e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:24:10.532926 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d04367-1004-48ff-a9f7-38bd71a993e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:24:10.534588 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d04367-1004-48ff-a9f7-38bd71a993e7 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:24:10.554240 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1072/2145] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:24:10 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:24:10.556199 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dde544-b0bf-42a2-9726-4fde8c7e7f6c admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:24:10.568402 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dde544-b0bf-42a2-9726-4fde8c7e7f6c admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:24:10.575948 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe865c16-cc4f-491d-ab93-53b9dc84eaab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:24:10.575948 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe865c16-cc4f-491d-ab93-53b9dc84eaab admin admin] REQUEST_METHOD: `PUT` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:24:10.576264 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe865c16-cc4f-491d-ab93-53b9dc84eaab admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:24:10.576348 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe865c16-cc4f-491d-ab93-53b9dc84eaab admin admin] PATH_INFO: `/v3/projects/9fc17d26483749619d08476678422476/users/b24920dfe0344057a440c17f947bf56f/roles/d009411496b8452b88e77ffb8db015a7` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:24:10.578705 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe865c16-cc4f-491d-ab93-53b9dc84eaab admin admin] RBAC: Authorizing `identity:create_grant(project_id=9fc17d26483749619d08476678422476, user_id=b24920dfe0344057a440c17f947bf56f, role_id=d009411496b8452b88e77ffb8db015a7)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:24:10.584575 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe865c16-cc4f-491d-ab93-53b9dc84eaab admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:24:10.605340 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1074/2146] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 01:24:10 2022] PUT /identity/v3/projects/9fc17d26483749619d08476678422476/users/b24920dfe0344057a440c17f947bf56f/roles/d009411496b8452b88e77ffb8db015a7 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:24:10.622807 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baf8872-af00-4b89-a6f1-0ee07715c7d0 None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:24:10.623019 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baf8872-af00-4b89-a6f1-0ee07715c7d0 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:24:10.623443 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baf8872-af00-4b89-a6f1-0ee07715c7d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:24:10.656347 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-8baf8872-af00-4b89-a6f1-0ee07715c7d0 None None] MFA Rules not processed for user `b24920dfe0344057a440c17f947bf56f`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:24:10.657734 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-8baf8872-af00-4b89-a6f1-0ee07715c7d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:24:10.690497 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:24:10.735001 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1073/2147] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:24:10 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:24:10.748701 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe865c16-cc4f-491d-ab93-53b9dc84eaab admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:24:10.750421 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-fe865c16-cc4f-491d-ab93-53b9dc84eaab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:24:10.773625 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:24:10.809535 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe865c16-cc4f-491d-ab93-53b9dc84eaab admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:24:10.813617 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47dd275d-662b-48a1-99a3-8046059ea28a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5c8fc69c518412ba1aad61a9f4f558c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:24:10.814263 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dd275d-662b-48a1-99a3-8046059ea28a service cinder] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:24:10.814551 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dd275d-662b-48a1-99a3-8046059ea28a service cinder] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:24:10.814827 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dd275d-662b-48a1-99a3-8046059ea28a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:24:10.824635 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47dd275d-662b-48a1-99a3-8046059ea28a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:24:10.826098 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47dd275d-662b-48a1-99a3-8046059ea28a service cinder] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:24:10.836278 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1075/2148] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:24:10 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:24:11.448797 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8baf8872-af00-4b89-a6f1-0ee07715c7d0 None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:24:11.515176 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8baf8872-af00-4b89-a6f1-0ee07715c7d0 None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:24:11.520066 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502f0381-a40b-499f-8e62-e8aa080cc941 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:24:11.523986 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502f0381-a40b-499f-8e62-e8aa080cc941 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:24:11.523986 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502f0381-a40b-499f-8e62-e8aa080cc941 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:24:11.523986 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502f0381-a40b-499f-8e62-e8aa080cc941 admin admin] PATH_INFO: `/v3/users/7865742f858940a78f838470daafe6e1` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:24:11.523986 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502f0381-a40b-499f-8e62-e8aa080cc941 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7865742f858940a78f838470daafe6e1)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:24:11.524487 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502f0381-a40b-499f-8e62-e8aa080cc941 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:24:11.736518 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-502f0381-a40b-499f-8e62-e8aa080cc941 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7865742f858940a78f838470daafe6e1'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:24:11.847036 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-502f0381-a40b-499f-8e62-e8aa080cc941 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7865742f858940a78f838470daafe6e1'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:24:11.847666 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-502f0381-a40b-499f-8e62-e8aa080cc941 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7865742f858940a78f838470daafe6e1'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:24:11.855270 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-502f0381-a40b-499f-8e62-e8aa080cc941 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7865742f858940a78f838470daafe6e1'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:24:11.855907 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-502f0381-a40b-499f-8e62-e8aa080cc941 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7865742f858940a78f838470daafe6e1'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:24:11.873313 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1074/2149] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 01:24:11 2022] DELETE /identity/v3/users/7865742f858940a78f838470daafe6e1 => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 01:24:11.877751 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e77897-9489-437c-a8c1-8a751eacb6c7 None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:24:11.878243 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e77897-9489-437c-a8c1-8a751eacb6c7 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:24:11.878336 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e77897-9489-437c-a8c1-8a751eacb6c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:24:11.898972 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-d4e77897-9489-437c-a8c1-8a751eacb6c7 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:24:11.900191 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-d4e77897-9489-437c-a8c1-8a751eacb6c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:24:11.963912 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1076/2150] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:24:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:24:11.973490 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502f0381-a40b-499f-8e62-e8aa080cc941 admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:24:11.974942 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-502f0381-a40b-499f-8e62-e8aa080cc941 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:24:12.030011 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:24:12.035462 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502f0381-a40b-499f-8e62-e8aa080cc941 admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:24:12.039558 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7770c22b-7d81-41dd-b53e-ce2e2540581b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0817c3320d5d41d8bba0550c8ae8dd60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:24:12.040197 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7770c22b-7d81-41dd-b53e-ce2e2540581b service neutron] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:24:12.040487 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7770c22b-7d81-41dd-b53e-ce2e2540581b service neutron] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:24:12.040851 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7770c22b-7d81-41dd-b53e-ce2e2540581b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:24:12.050707 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7770c22b-7d81-41dd-b53e-ce2e2540581b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:24:12.052333 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7770c22b-7d81-41dd-b53e-ce2e2540581b service neutron] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:24:12.061761 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1075/2151] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:24:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:24:12.533864 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e77897-9489-437c-a8c1-8a751eacb6c7 None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:24:12.535383 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-d4e77897-9489-437c-a8c1-8a751eacb6c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:24:12.550160 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e77897-9489-437c-a8c1-8a751eacb6c7 None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:24:12.555549 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16873c2-c9d0-42d4-8c75-69d529699012 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:24:12.556221 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16873c2-c9d0-42d4-8c75-69d529699012 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:24:12.556507 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16873c2-c9d0-42d4-8c75-69d529699012 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:24:12.556827 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16873c2-c9d0-42d4-8c75-69d529699012 admin admin] PATH_INFO: `/v3/projects/e95043f03e76408ea9613c61faf2b8b5` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:24:12.558608 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16873c2-c9d0-42d4-8c75-69d529699012 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e95043f03e76408ea9613c61faf2b8b5)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:24:12.560794 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16873c2-c9d0-42d4-8c75-69d529699012 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:24:12.689551 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-b16873c2-c9d0-42d4-8c75-69d529699012 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e95043f03e76408ea9613c61faf2b8b5', 'request_id': 'req-b16873c2-c9d0-42d4-8c75-69d529699012'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:24:12.696960 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-b16873c2-c9d0-42d4-8c75-69d529699012 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e95043f03e76408ea9613c61faf2b8b5', 'request_id': 'req-b16873c2-c9d0-42d4-8c75-69d529699012'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:24:12.778778 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-b16873c2-c9d0-42d4-8c75-69d529699012 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e95043f03e76408ea9613c61faf2b8b5', 'request_id': 'req-b16873c2-c9d0-42d4-8c75-69d529699012'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:24:12.791844 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-b16873c2-c9d0-42d4-8c75-69d529699012 admin admin] The token cache is being invalidate because project e95043f03e76408ea9613c61faf2b8b5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81750) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 01:24:12.792143 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-b16873c2-c9d0-42d4-8c75-69d529699012 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:24:12.792847 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-b16873c2-c9d0-42d4-8c75-69d529699012 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:24:12.794529 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1077/2152] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 01:24:12 2022] DELETE /identity/v3/projects/e95043f03e76408ea9613c61faf2b8b5 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:24:12.798645 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7770c22b-7d81-41dd-b53e-ce2e2540581b service neutron] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:24:12.809589 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7770c22b-7d81-41dd-b53e-ce2e2540581b service neutron] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:24:12.868294 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11096802-16f3-44eb-9b27-ad326e093175 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:24:12.868958 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11096802-16f3-44eb-9b27-ad326e093175 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:24:12.869244 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11096802-16f3-44eb-9b27-ad326e093175 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:24:12.869518 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11096802-16f3-44eb-9b27-ad326e093175 admin admin] PATH_INFO: `/v3/users/b24920dfe0344057a440c17f947bf56f` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:24:12.870623 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11096802-16f3-44eb-9b27-ad326e093175 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b24920dfe0344057a440c17f947bf56f)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:24:12.872569 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11096802-16f3-44eb-9b27-ad326e093175 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:24:13.042791 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-11096802-16f3-44eb-9b27-ad326e093175 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b24920dfe0344057a440c17f947bf56f'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:24:13.108743 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-11096802-16f3-44eb-9b27-ad326e093175 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b24920dfe0344057a440c17f947bf56f'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:24:13.109836 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-11096802-16f3-44eb-9b27-ad326e093175 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b24920dfe0344057a440c17f947bf56f'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:24:13.124132 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-11096802-16f3-44eb-9b27-ad326e093175 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b24920dfe0344057a440c17f947bf56f'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:24:13.125072 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-11096802-16f3-44eb-9b27-ad326e093175 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b24920dfe0344057a440c17f947bf56f'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:24:13.149507 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1076/2153] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 01:24:12 2022] DELETE /identity/v3/users/b24920dfe0344057a440c17f947bf56f => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:24:13.634517 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16873c2-c9d0-42d4-8c75-69d529699012 admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:24:13.634517 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-b16873c2-c9d0-42d4-8c75-69d529699012 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:24:13.659666 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16873c2-c9d0-42d4-8c75-69d529699012 admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:24:13.729949 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:24:13.751198 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a24b49-7609-46e5-af31-ac739aa69cfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:24:13.751922 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a24b49-7609-46e5-af31-ac739aa69cfe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:24:13.752210 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a24b49-7609-46e5-af31-ac739aa69cfe admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:24:13.752491 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a24b49-7609-46e5-af31-ac739aa69cfe admin admin] PATH_INFO: `/v3/projects/9fc17d26483749619d08476678422476` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:24:13.753723 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a24b49-7609-46e5-af31-ac739aa69cfe admin admin] RBAC: Authorizing `identity:delete_project(project_id=9fc17d26483749619d08476678422476)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:24:13.755705 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a24b49-7609-46e5-af31-ac739aa69cfe admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:24:13.916069 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-10a24b49-7609-46e5-af31-ac739aa69cfe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9fc17d26483749619d08476678422476', 'request_id': 'req-10a24b49-7609-46e5-af31-ac739aa69cfe'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:24:13.925062 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-10a24b49-7609-46e5-af31-ac739aa69cfe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9fc17d26483749619d08476678422476', 'request_id': 'req-10a24b49-7609-46e5-af31-ac739aa69cfe'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:24:13.984912 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-10a24b49-7609-46e5-af31-ac739aa69cfe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9fc17d26483749619d08476678422476', 'request_id': 'req-10a24b49-7609-46e5-af31-ac739aa69cfe'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:24:13.998525 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-10a24b49-7609-46e5-af31-ac739aa69cfe admin admin] The token cache is being invalidate because project 9fc17d26483749619d08476678422476 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81750) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 01:24:13.998821 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-10a24b49-7609-46e5-af31-ac739aa69cfe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:24:13.999804 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-10a24b49-7609-46e5-af31-ac739aa69cfe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:24:14.001523 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1078/2154] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 01:24:13 2022] DELETE /identity/v3/projects/9fc17d26483749619d08476678422476 => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:24:14.007769 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08459575-b294-49c2-83d1-a3de9a4d5788 None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:24:14.008226 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08459575-b294-49c2-83d1-a3de9a4d5788 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:24:14.008466 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08459575-b294-49c2-83d1-a3de9a4d5788 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:24:14.032707 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-08459575-b294-49c2-83d1-a3de9a4d5788 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:24:14.034145 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-08459575-b294-49c2-83d1-a3de9a4d5788 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:24:14.108200 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1077/2155] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:24:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:24:14.115741 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5702f13-ee34-4291-a964-b32416264cbf None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:24:14.116031 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5702f13-ee34-4291-a964-b32416264cbf None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:24:14.116306 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5702f13-ee34-4291-a964-b32416264cbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:24:14.139404 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-d5702f13-ee34-4291-a964-b32416264cbf None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:24:14.142003 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-d5702f13-ee34-4291-a964-b32416264cbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:24:14.154360 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1079/2156] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:24:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:24:14.158669 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08459575-b294-49c2-83d1-a3de9a4d5788 None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:24:14.168747 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08459575-b294-49c2-83d1-a3de9a4d5788 None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:24:14.173747 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548ea18e-72ed-42f3-a109-99d06a270292 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:24:14.174373 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548ea18e-72ed-42f3-a109-99d06a270292 admin admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:24:14.174655 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548ea18e-72ed-42f3-a109-99d06a270292 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:24:14.174932 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548ea18e-72ed-42f3-a109-99d06a270292 admin admin] PATH_INFO: `/v3/domains` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:24:14.175591 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548ea18e-72ed-42f3-a109-99d06a270292 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:24:14.175902 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548ea18e-72ed-42f3-a109-99d06a270292 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:24:14.177856 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548ea18e-72ed-42f3-a109-99d06a270292 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:24:14.185386 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:24:14.190819 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1078/2157] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Aug 31 01:24:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:24:14.198014 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5702f13-ee34-4291-a964-b32416264cbf None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:24:14.211496 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5702f13-ee34-4291-a964-b32416264cbf None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:24:14.213848 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973dbb18-68f3-4c71-912f-5fbeb83e8a19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:24:14.214686 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973dbb18-68f3-4c71-912f-5fbeb83e8a19 admin admin] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:24:14.215014 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973dbb18-68f3-4c71-912f-5fbeb83e8a19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:24:14.215290 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973dbb18-68f3-4c71-912f-5fbeb83e8a19 admin admin] PATH_INFO: `/v3/projects` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:24:14.215926 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973dbb18-68f3-4c71-912f-5fbeb83e8a19 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:24:14.217950 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973dbb18-68f3-4c71-912f-5fbeb83e8a19 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:24:14.331825 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1080/2158] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 01:24:14 2022] POST /identity/v3/projects => generated 367 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:24:14.333025 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548ea18e-72ed-42f3-a109-99d06a270292 admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:24:14.343332 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548ea18e-72ed-42f3-a109-99d06a270292 admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:24:14.401969 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2ca713-8273-455e-88e2-b2e36e826a0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:24:14.402580 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2ca713-8273-455e-88e2-b2e36e826a0e admin admin] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:24:14.402865 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2ca713-8273-455e-88e2-b2e36e826a0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:24:14.403141 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2ca713-8273-455e-88e2-b2e36e826a0e admin admin] PATH_INFO: `/v3/users` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:24:14.403779 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2ca713-8273-455e-88e2-b2e36e826a0e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:24:14.405878 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2ca713-8273-455e-88e2-b2e36e826a0e admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:24:14.497037 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1079/2159] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 01:24:14 2022] POST /identity/v3/users => generated 284 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 01:24:14.501221 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973dbb18-68f3-4c71-912f-5fbeb83e8a19 admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:24:14.511525 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973dbb18-68f3-4c71-912f-5fbeb83e8a19 admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:24:14.517082 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8537f04b-9cff-4a10-bce4-8eb70a0b6b39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:24:14.517740 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8537f04b-9cff-4a10-bce4-8eb70a0b6b39 admin admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:24:14.520710 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8537f04b-9cff-4a10-bce4-8eb70a0b6b39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:24:14.520710 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8537f04b-9cff-4a10-bce4-8eb70a0b6b39 admin admin] PATH_INFO: `/v3/roles` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:24:14.520710 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8537f04b-9cff-4a10-bce4-8eb70a0b6b39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:24:14.521329 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8537f04b-9cff-4a10-bce4-8eb70a0b6b39 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:24:14.531728 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1081/2160] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:24:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:24:14.535649 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2ca713-8273-455e-88e2-b2e36e826a0e admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:24:14.546429 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2ca713-8273-455e-88e2-b2e36e826a0e admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:24:14.551171 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5899fd0-7910-4108-a4a4-7378d548e497 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:24:14.551766 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5899fd0-7910-4108-a4a4-7378d548e497 admin admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:24:14.552049 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5899fd0-7910-4108-a4a4-7378d548e497 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:24:14.552441 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5899fd0-7910-4108-a4a4-7378d548e497 admin admin] PATH_INFO: `/v3/roles` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:24:14.553109 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5899fd0-7910-4108-a4a4-7378d548e497 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:24:14.554707 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5899fd0-7910-4108-a4a4-7378d548e497 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:24:14.569189 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1080/2161] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:24:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:24:14.570465 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8537f04b-9cff-4a10-bce4-8eb70a0b6b39 admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:24:14.581103 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8537f04b-9cff-4a10-bce4-8eb70a0b6b39 admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:24:14.586737 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e893bf16-082b-457a-bcab-1990455582d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:24:14.587424 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e893bf16-082b-457a-bcab-1990455582d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:24:14.587701 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e893bf16-082b-457a-bcab-1990455582d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:24:14.588078 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e893bf16-082b-457a-bcab-1990455582d7 admin admin] PATH_INFO: `/v3/projects/f93acac1ed504852b8239acc71779630/users/8a3fb0c04c874d37b66916032680556b/roles/d0b9768f4d014766b9326387f2e11f34` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:24:14.604453 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e893bf16-082b-457a-bcab-1990455582d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f93acac1ed504852b8239acc71779630, user_id=8a3fb0c04c874d37b66916032680556b, role_id=d0b9768f4d014766b9326387f2e11f34)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:24:14.607160 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e893bf16-082b-457a-bcab-1990455582d7 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:24:14.703962 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1082/2162] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 01:24:14 2022] PUT /identity/v3/projects/f93acac1ed504852b8239acc71779630/users/8a3fb0c04c874d37b66916032680556b/roles/d0b9768f4d014766b9326387f2e11f34 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:24:14.714606 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a540090b-ae49-4d41-a5ad-8ebf55498bcd None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:24:14.714900 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a540090b-ae49-4d41-a5ad-8ebf55498bcd None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:24:14.715239 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a540090b-ae49-4d41-a5ad-8ebf55498bcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:24:14.746408 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-a540090b-ae49-4d41-a5ad-8ebf55498bcd None None] MFA Rules not processed for user `8a3fb0c04c874d37b66916032680556b`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:24:14.747828 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-a540090b-ae49-4d41-a5ad-8ebf55498bcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:24:14.817433 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1081/2163] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:24:14 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:24:14.830040 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e893bf16-082b-457a-bcab-1990455582d7 admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:24:14.831819 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-e893bf16-082b-457a-bcab-1990455582d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:24:14.892856 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e893bf16-082b-457a-bcab-1990455582d7 admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:24:14.897453 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f20aa44-6e13-4e6f-91fb-ed1fa4a86126 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8590c288028418ebe7a4163fd8039f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:24:14.900128 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f20aa44-6e13-4e6f-91fb-ed1fa4a86126 service glance] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:24:14.900478 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f20aa44-6e13-4e6f-91fb-ed1fa4a86126 service glance] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:24:14.901038 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f20aa44-6e13-4e6f-91fb-ed1fa4a86126 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:24:14.910816 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f20aa44-6e13-4e6f-91fb-ed1fa4a86126 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:24:14.922396 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f20aa44-6e13-4e6f-91fb-ed1fa4a86126 service glance] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:24:14.929019 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1083/2164] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:24:14 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:24:19.220955 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a540090b-ae49-4d41-a5ad-8ebf55498bcd None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:24:19.222427 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-a540090b-ae49-4d41-a5ad-8ebf55498bcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:24:19.296779 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a540090b-ae49-4d41-a5ad-8ebf55498bcd None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:24:19.301833 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e997c2e-0990-4008-9229-16c79de537c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:24:19.302464 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e997c2e-0990-4008-9229-16c79de537c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:24:19.302750 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e997c2e-0990-4008-9229-16c79de537c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:24:19.303032 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e997c2e-0990-4008-9229-16c79de537c5 admin admin] PATH_INFO: `/v3/users/0833316657274dda842a118a79fba788` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:24:19.304161 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e997c2e-0990-4008-9229-16c79de537c5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0833316657274dda842a118a79fba788)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:24:19.306142 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e997c2e-0990-4008-9229-16c79de537c5 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:24:19.314425 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:24:19.466517 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-5e997c2e-0990-4008-9229-16c79de537c5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0833316657274dda842a118a79fba788'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:24:19.532936 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-5e997c2e-0990-4008-9229-16c79de537c5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0833316657274dda842a118a79fba788'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:24:19.533546 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-5e997c2e-0990-4008-9229-16c79de537c5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0833316657274dda842a118a79fba788'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:24:19.553559 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-5e997c2e-0990-4008-9229-16c79de537c5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0833316657274dda842a118a79fba788'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:24:19.554309 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-5e997c2e-0990-4008-9229-16c79de537c5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0833316657274dda842a118a79fba788'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:24:19.577307 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1082/2165] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 01:24:19 2022] DELETE /identity/v3/users/0833316657274dda842a118a79fba788 => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:24:19.582464 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63069c1f-3a09-4955-9690-b00ca7cb9faa None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:24:19.584858 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63069c1f-3a09-4955-9690-b00ca7cb9faa None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:24:19.585293 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63069c1f-3a09-4955-9690-b00ca7cb9faa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:24:19.605794 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-63069c1f-3a09-4955-9690-b00ca7cb9faa None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:24:19.607046 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-63069c1f-3a09-4955-9690-b00ca7cb9faa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:24:19.620462 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:24:19.671080 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1084/2166] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:24:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:24:19.681695 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e997c2e-0990-4008-9229-16c79de537c5 admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:24:19.687848 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-5e997c2e-0990-4008-9229-16c79de537c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:24:19.747410 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e997c2e-0990-4008-9229-16c79de537c5 admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:24:19.751424 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbf2a78-3c12-4de5-a190-3598d57546b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0817c3320d5d41d8bba0550c8ae8dd60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:24:19.752821 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbf2a78-3c12-4de5-a190-3598d57546b2 service neutron] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:24:19.753477 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbf2a78-3c12-4de5-a190-3598d57546b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:24:19.754332 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbf2a78-3c12-4de5-a190-3598d57546b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:24:19.775982 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbf2a78-3c12-4de5-a190-3598d57546b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:24:19.777528 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbf2a78-3c12-4de5-a190-3598d57546b2 service neutron] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:24:19.790598 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1083/2167] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:24:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:24:20.242420 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63069c1f-3a09-4955-9690-b00ca7cb9faa None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:24:20.244116 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-63069c1f-3a09-4955-9690-b00ca7cb9faa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:24:20.257419 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63069c1f-3a09-4955-9690-b00ca7cb9faa None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:24:20.263069 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d856da92-fb10-44dc-a90b-748ed36ed034 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:24:20.263878 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d856da92-fb10-44dc-a90b-748ed36ed034 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:24:20.264285 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d856da92-fb10-44dc-a90b-748ed36ed034 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:24:20.264670 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d856da92-fb10-44dc-a90b-748ed36ed034 admin admin] PATH_INFO: `/v3/projects/3d340ee4f1f74140aa321c30ed1b9456` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:24:20.270398 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d856da92-fb10-44dc-a90b-748ed36ed034 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3d340ee4f1f74140aa321c30ed1b9456)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:24:20.272426 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d856da92-fb10-44dc-a90b-748ed36ed034 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:24:20.394388 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-d856da92-fb10-44dc-a90b-748ed36ed034 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3d340ee4f1f74140aa321c30ed1b9456', 'request_id': 'req-d856da92-fb10-44dc-a90b-748ed36ed034'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:24:20.408223 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-d856da92-fb10-44dc-a90b-748ed36ed034 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3d340ee4f1f74140aa321c30ed1b9456', 'request_id': 'req-d856da92-fb10-44dc-a90b-748ed36ed034'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:24:20.470639 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-d856da92-fb10-44dc-a90b-748ed36ed034 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3d340ee4f1f74140aa321c30ed1b9456', 'request_id': 'req-d856da92-fb10-44dc-a90b-748ed36ed034'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:24:20.483197 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-d856da92-fb10-44dc-a90b-748ed36ed034 admin admin] The token cache is being invalidate because project 3d340ee4f1f74140aa321c30ed1b9456 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81750) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 01:24:20.483497 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-d856da92-fb10-44dc-a90b-748ed36ed034 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:24:20.484603 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-d856da92-fb10-44dc-a90b-748ed36ed034 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:24:20.486352 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1085/2168] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 01:24:20 2022] DELETE /identity/v3/projects/3d340ee4f1f74140aa321c30ed1b9456 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:24:20.846028 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbf2a78-3c12-4de5-a190-3598d57546b2 service neutron] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:24:20.848118 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-4bbf2a78-3c12-4de5-a190-3598d57546b2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:24:20.863092 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbf2a78-3c12-4de5-a190-3598d57546b2 service neutron] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:24:20.933588 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28543b70-8e56-4fa3-9e9e-cab1153b2c87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:24:20.934329 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28543b70-8e56-4fa3-9e9e-cab1153b2c87 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:24:20.934682 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28543b70-8e56-4fa3-9e9e-cab1153b2c87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:24:20.935030 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28543b70-8e56-4fa3-9e9e-cab1153b2c87 admin admin] PATH_INFO: `/v3/users/c58de488767a410a9d7de3a541fc8d88` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:24:20.936345 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28543b70-8e56-4fa3-9e9e-cab1153b2c87 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c58de488767a410a9d7de3a541fc8d88)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:24:20.938482 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28543b70-8e56-4fa3-9e9e-cab1153b2c87 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:24:21.169005 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:24:21.188106 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-28543b70-8e56-4fa3-9e9e-cab1153b2c87 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c58de488767a410a9d7de3a541fc8d88'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:24:21.281260 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-28543b70-8e56-4fa3-9e9e-cab1153b2c87 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c58de488767a410a9d7de3a541fc8d88'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:24:21.283072 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-28543b70-8e56-4fa3-9e9e-cab1153b2c87 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c58de488767a410a9d7de3a541fc8d88'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:24:21.296306 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-28543b70-8e56-4fa3-9e9e-cab1153b2c87 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c58de488767a410a9d7de3a541fc8d88'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:24:21.297597 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-28543b70-8e56-4fa3-9e9e-cab1153b2c87 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c58de488767a410a9d7de3a541fc8d88'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:24:21.319792 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1084/2169] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 01:24:20 2022] DELETE /identity/v3/users/c58de488767a410a9d7de3a541fc8d88 => generated 0 bytes in 474 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:24:21.324120 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e71f65-eceb-4d86-9693-b0b57e2997f5 None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:24:21.324894 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e71f65-eceb-4d86-9693-b0b57e2997f5 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:24:21.324894 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e71f65-eceb-4d86-9693-b0b57e2997f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:24:21.349086 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-89e71f65-eceb-4d86-9693-b0b57e2997f5 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:24:21.350314 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-89e71f65-eceb-4d86-9693-b0b57e2997f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:24:21.410895 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1086/2170] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:24:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:24:21.421273 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28543b70-8e56-4fa3-9e9e-cab1153b2c87 admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:24:21.422838 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-28543b70-8e56-4fa3-9e9e-cab1153b2c87 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:24:21.480174 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51defc3-498f-408a-9cb5-edf2553a9133 None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:24:21.480597 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51defc3-498f-408a-9cb5-edf2553a9133 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:24:21.481015 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51defc3-498f-408a-9cb5-edf2553a9133 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:24:21.486308 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28543b70-8e56-4fa3-9e9e-cab1153b2c87 admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:24:21.490062 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3342713a-1436-4756-9f53-23fc74f08e1a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0817c3320d5d41d8bba0550c8ae8dd60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:24:21.490681 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3342713a-1436-4756-9f53-23fc74f08e1a service neutron] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:24:21.490966 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3342713a-1436-4756-9f53-23fc74f08e1a service neutron] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:24:21.491241 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3342713a-1436-4756-9f53-23fc74f08e1a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:24:21.501104 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-c51defc3-498f-408a-9cb5-edf2553a9133 None None] MFA Rules not processed for user `71d0408df86f4be0ac59cc80575a14e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:24:21.501713 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3342713a-1436-4756-9f53-23fc74f08e1a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:24:21.502681 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-c51defc3-498f-408a-9cb5-edf2553a9133 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:24:21.503266 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3342713a-1436-4756-9f53-23fc74f08e1a service neutron] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:24:21.516482 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1085/2171] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:24:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:24:21.543940 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:24:21.571047 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1087/2172] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:24:21 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 01:24:22.014899 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3342713a-1436-4756-9f53-23fc74f08e1a service neutron] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:24:22.016962 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-3342713a-1436-4756-9f53-23fc74f08e1a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:24:22.045892 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3342713a-1436-4756-9f53-23fc74f08e1a service neutron] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:24:22.054028 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42a734f-4954-45aa-bb5b-b8a208a95016 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:24:22.054791 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42a734f-4954-45aa-bb5b-b8a208a95016 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:24:22.055307 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42a734f-4954-45aa-bb5b-b8a208a95016 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:24:22.055660 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42a734f-4954-45aa-bb5b-b8a208a95016 admin admin] PATH_INFO: `/v3/projects/db5c6269a518420088c3293504575903` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:24:22.059132 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42a734f-4954-45aa-bb5b-b8a208a95016 admin admin] RBAC: Authorizing `identity:delete_project(project_id=db5c6269a518420088c3293504575903)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:24:22.061170 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42a734f-4954-45aa-bb5b-b8a208a95016 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:24:22.227292 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-a42a734f-4954-45aa-bb5b-b8a208a95016 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'db5c6269a518420088c3293504575903', 'request_id': 'req-a42a734f-4954-45aa-bb5b-b8a208a95016'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:24:22.235368 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-a42a734f-4954-45aa-bb5b-b8a208a95016 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'db5c6269a518420088c3293504575903', 'request_id': 'req-a42a734f-4954-45aa-bb5b-b8a208a95016'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:24:22.256038 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-a42a734f-4954-45aa-bb5b-b8a208a95016 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'db5c6269a518420088c3293504575903', 'request_id': 'req-a42a734f-4954-45aa-bb5b-b8a208a95016'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:24:22.345141 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-a42a734f-4954-45aa-bb5b-b8a208a95016 admin admin] The token cache is being invalidate because project db5c6269a518420088c3293504575903 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81749) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 01:24:22.345557 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-a42a734f-4954-45aa-bb5b-b8a208a95016 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:24:22.349731 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-a42a734f-4954-45aa-bb5b-b8a208a95016 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:24:22.353857 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849b93f4-5f26-4454-ba68-e7adb840cbb9 None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:24:22.354379 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849b93f4-5f26-4454-ba68-e7adb840cbb9 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:24:22.354603 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1086/2173] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 01:24:22 2022] DELETE /identity/v3/projects/db5c6269a518420088c3293504575903 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:24:22.354697 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849b93f4-5f26-4454-ba68-e7adb840cbb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:24:22.401279 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-849b93f4-5f26-4454-ba68-e7adb840cbb9 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:24:22.407201 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-849b93f4-5f26-4454-ba68-e7adb840cbb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:24:22.486842 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1088/2174] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:24:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 01:24:22.496087 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c942f4c6-bb2f-473d-89e1-c472840b28f4 None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:24:22.496445 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c942f4c6-bb2f-473d-89e1-c472840b28f4 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:24:22.497017 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c942f4c6-bb2f-473d-89e1-c472840b28f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:24:22.526974 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-c942f4c6-bb2f-473d-89e1-c472840b28f4 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:24:22.528892 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-c942f4c6-bb2f-473d-89e1-c472840b28f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:24:22.538503 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1087/2175] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:24:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:24:22.544605 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849b93f4-5f26-4454-ba68-e7adb840cbb9 None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:24:22.557296 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849b93f4-5f26-4454-ba68-e7adb840cbb9 None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:24:22.566083 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07ec108-7c63-4cf6-a951-f6c213003a91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:24:22.567593 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07ec108-7c63-4cf6-a951-f6c213003a91 admin admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:24:22.567947 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07ec108-7c63-4cf6-a951-f6c213003a91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:24:22.568296 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07ec108-7c63-4cf6-a951-f6c213003a91 admin admin] PATH_INFO: `/v3/domains` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:24:22.569633 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07ec108-7c63-4cf6-a951-f6c213003a91 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:24:22.570020 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07ec108-7c63-4cf6-a951-f6c213003a91 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:24:22.572994 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07ec108-7c63-4cf6-a951-f6c213003a91 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:24:22.591224 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1089/2176] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Aug 31 01:24:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:24:22.596210 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c942f4c6-bb2f-473d-89e1-c472840b28f4 None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:24:22.605765 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c942f4c6-bb2f-473d-89e1-c472840b28f4 None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:24:22.611575 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb868173-24f8-4f8d-a38c-e8bd5c5d76cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:24:22.618712 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb868173-24f8-4f8d-a38c-e8bd5c5d76cd admin admin] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:24:22.618712 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb868173-24f8-4f8d-a38c-e8bd5c5d76cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:24:22.618712 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb868173-24f8-4f8d-a38c-e8bd5c5d76cd admin admin] PATH_INFO: `/v3/projects` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:24:22.618712 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb868173-24f8-4f8d-a38c-e8bd5c5d76cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:24:22.618712 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb868173-24f8-4f8d-a38c-e8bd5c5d76cd admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:24:22.740176 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07ec108-7c63-4cf6-a951-f6c213003a91 admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:24:22.746062 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1088/2177] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 01:24:22 2022] POST /identity/v3/projects => generated 375 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:24:22.760610 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07ec108-7c63-4cf6-a951-f6c213003a91 admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:24:22.864822 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9493392b-6a70-4753-9241-541fa7ad1f41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:24:22.865536 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9493392b-6a70-4753-9241-541fa7ad1f41 admin admin] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:24:22.865910 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9493392b-6a70-4753-9241-541fa7ad1f41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:24:22.866260 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9493392b-6a70-4753-9241-541fa7ad1f41 admin admin] PATH_INFO: `/v3/users` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:24:22.867007 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9493392b-6a70-4753-9241-541fa7ad1f41 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:24:22.869114 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9493392b-6a70-4753-9241-541fa7ad1f41 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:24:22.957456 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1090/2178] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 01:24:22 2022] POST /identity/v3/users => generated 288 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 01:24:22.961930 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb868173-24f8-4f8d-a38c-e8bd5c5d76cd admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:24:22.969082 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:24:22.993357 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb868173-24f8-4f8d-a38c-e8bd5c5d76cd admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:24:23.012302 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383dc1ec-3c65-418c-932f-6ce5ab255a87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:24:23.012947 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383dc1ec-3c65-418c-932f-6ce5ab255a87 admin admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:24:23.013228 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383dc1ec-3c65-418c-932f-6ce5ab255a87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:24:23.016452 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383dc1ec-3c65-418c-932f-6ce5ab255a87 admin admin] PATH_INFO: `/v3/roles` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:24:23.016452 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383dc1ec-3c65-418c-932f-6ce5ab255a87 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:24:23.016452 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383dc1ec-3c65-418c-932f-6ce5ab255a87 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:24:23.042533 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1089/2179] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:24:22 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:24:23.049929 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9493392b-6a70-4753-9241-541fa7ad1f41 admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:24:23.067167 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9493392b-6a70-4753-9241-541fa7ad1f41 admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:24:23.075283 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b9f996-c0ba-4a1b-9d55-11e2be0fd9c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:24:23.075910 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b9f996-c0ba-4a1b-9d55-11e2be0fd9c3 admin admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:24:23.076533 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b9f996-c0ba-4a1b-9d55-11e2be0fd9c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:24:23.076533 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b9f996-c0ba-4a1b-9d55-11e2be0fd9c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:24:23.077957 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b9f996-c0ba-4a1b-9d55-11e2be0fd9c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:24:23.080297 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b9f996-c0ba-4a1b-9d55-11e2be0fd9c3 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:24:23.099370 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1091/2180] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:24:23 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 01:24:23.100569 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383dc1ec-3c65-418c-932f-6ce5ab255a87 admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:24:23.113734 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383dc1ec-3c65-418c-932f-6ce5ab255a87 admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:24:23.118592 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baff1952-db23-4382-b864-854f00fc6e4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:24:23.119537 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baff1952-db23-4382-b864-854f00fc6e4d admin admin] REQUEST_METHOD: `PUT` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:24:23.119827 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baff1952-db23-4382-b864-854f00fc6e4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:24:23.120106 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baff1952-db23-4382-b864-854f00fc6e4d admin admin] PATH_INFO: `/v3/projects/5f525dac532047f29df0eb7337b99d54/users/5c6221b2bb1a49e1aefb9b64e6aab03b/roles/d0b9768f4d014766b9326387f2e11f34` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:24:23.146047 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baff1952-db23-4382-b864-854f00fc6e4d admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f525dac532047f29df0eb7337b99d54, user_id=5c6221b2bb1a49e1aefb9b64e6aab03b, role_id=d0b9768f4d014766b9326387f2e11f34)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:24:23.149081 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baff1952-db23-4382-b864-854f00fc6e4d admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:24:23.246031 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1090/2181] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 01:24:23 2022] PUT /identity/v3/projects/5f525dac532047f29df0eb7337b99d54/users/5c6221b2bb1a49e1aefb9b64e6aab03b/roles/d0b9768f4d014766b9326387f2e11f34 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:24:23.253010 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89fb1f4-97c1-4cd7-ac34-9a24681f579d None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:24:23.253477 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89fb1f4-97c1-4cd7-ac34-9a24681f579d None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:24:23.254436 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89fb1f4-97c1-4cd7-ac34-9a24681f579d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:24:23.288443 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-c89fb1f4-97c1-4cd7-ac34-9a24681f579d None None] MFA Rules not processed for user `5c6221b2bb1a49e1aefb9b64e6aab03b`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:24:23.289791 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-c89fb1f4-97c1-4cd7-ac34-9a24681f579d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:24:23.347694 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:24:23.383379 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1092/2182] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:24:23 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:24:23.391179 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baff1952-db23-4382-b864-854f00fc6e4d admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:24:23.397376 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-baff1952-db23-4382-b864-854f00fc6e4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:24:23.464467 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baff1952-db23-4382-b864-854f00fc6e4d admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:24:23.476123 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f040d4cc-d3e9-4724-b56a-d2169ab6f399 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5c8fc69c518412ba1aad61a9f4f558c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:24:23.477002 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f040d4cc-d3e9-4724-b56a-d2169ab6f399 service cinder] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:24:23.477684 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f040d4cc-d3e9-4724-b56a-d2169ab6f399 service cinder] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:24:23.477684 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f040d4cc-d3e9-4724-b56a-d2169ab6f399 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:24:23.487687 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f040d4cc-d3e9-4724-b56a-d2169ab6f399 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:24:23.489921 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f040d4cc-d3e9-4724-b56a-d2169ab6f399 service cinder] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:24:23.501686 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1091/2183] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:24:23 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:24:25.106075 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed59a38e-584c-493f-a9b1-471f36d311ad None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:24:25.107106 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed59a38e-584c-493f-a9b1-471f36d311ad None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:24:25.107106 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed59a38e-584c-493f-a9b1-471f36d311ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:24:25.125794 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-ed59a38e-584c-493f-a9b1-471f36d311ad None None] MFA Rules not processed for user `52ed99737fd240789bb682fd9bf11be3`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:24:25.126943 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-ed59a38e-584c-493f-a9b1-471f36d311ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:24:25.183997 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1093/2184] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:24:25 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:24:33.495850 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=39,keystone:INSERT=1 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:24:35.176913 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=34 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:24:51.636265 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f040d4cc-d3e9-4724-b56a-d2169ab6f399 service cinder] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:24:51.706493 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f040d4cc-d3e9-4724-b56a-d2169ab6f399 service cinder] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:24:51.711561 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2cdfaa-82f4-4cec-99fa-9184abad0662 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:24:51.712226 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2cdfaa-82f4-4cec-99fa-9184abad0662 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:24:51.712513 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2cdfaa-82f4-4cec-99fa-9184abad0662 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:24:51.712818 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2cdfaa-82f4-4cec-99fa-9184abad0662 admin admin] PATH_INFO: `/v3/users/5c6221b2bb1a49e1aefb9b64e6aab03b` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:24:51.713988 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2cdfaa-82f4-4cec-99fa-9184abad0662 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5c6221b2bb1a49e1aefb9b64e6aab03b)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:24:51.715918 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2cdfaa-82f4-4cec-99fa-9184abad0662 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:24:51.896131 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-0e2cdfaa-82f4-4cec-99fa-9184abad0662 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5c6221b2bb1a49e1aefb9b64e6aab03b'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:24:51.954547 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-0e2cdfaa-82f4-4cec-99fa-9184abad0662 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5c6221b2bb1a49e1aefb9b64e6aab03b'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:24:51.955280 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-0e2cdfaa-82f4-4cec-99fa-9184abad0662 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5c6221b2bb1a49e1aefb9b64e6aab03b'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:24:51.964052 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-0e2cdfaa-82f4-4cec-99fa-9184abad0662 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5c6221b2bb1a49e1aefb9b64e6aab03b'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:24:51.964924 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-0e2cdfaa-82f4-4cec-99fa-9184abad0662 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5c6221b2bb1a49e1aefb9b64e6aab03b'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:24:51.982945 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1092/2185] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 01:24:51 2022] DELETE /identity/v3/users/5c6221b2bb1a49e1aefb9b64e6aab03b => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:24:51.987370 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966c8575-a168-40d1-b077-8e4d11f2ec9e None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:24:51.987872 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966c8575-a168-40d1-b077-8e4d11f2ec9e None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:24:51.987967 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966c8575-a168-40d1-b077-8e4d11f2ec9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:24:52.008514 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-966c8575-a168-40d1-b077-8e4d11f2ec9e None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:24:52.009903 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-966c8575-a168-40d1-b077-8e4d11f2ec9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:24:52.070602 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1094/2186] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:24:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:24:52.081393 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2cdfaa-82f4-4cec-99fa-9184abad0662 admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:24:52.083557 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-0e2cdfaa-82f4-4cec-99fa-9184abad0662 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:24:52.148344 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2cdfaa-82f4-4cec-99fa-9184abad0662 admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:24:52.152129 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d69822-4258-4d74-85fe-3f18c82211ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0817c3320d5d41d8bba0550c8ae8dd60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:24:52.152782 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d69822-4258-4d74-85fe-3f18c82211ae service neutron] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:24:52.153242 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d69822-4258-4d74-85fe-3f18c82211ae service neutron] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:24:52.153526 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d69822-4258-4d74-85fe-3f18c82211ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:24:52.162833 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d69822-4258-4d74-85fe-3f18c82211ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:24:52.164309 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d69822-4258-4d74-85fe-3f18c82211ae service neutron] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:24:52.174242 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1093/2187] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:24:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:24:52.584692 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966c8575-a168-40d1-b077-8e4d11f2ec9e None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:24:52.586561 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-966c8575-a168-40d1-b077-8e4d11f2ec9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:24:52.602936 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966c8575-a168-40d1-b077-8e4d11f2ec9e None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:24:52.608755 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a46bd1-07d3-49d7-86f9-4a303a750675 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:24:52.609470 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a46bd1-07d3-49d7-86f9-4a303a750675 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:24:52.609983 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a46bd1-07d3-49d7-86f9-4a303a750675 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:24:52.610556 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a46bd1-07d3-49d7-86f9-4a303a750675 admin admin] PATH_INFO: `/v3/projects/5f525dac532047f29df0eb7337b99d54` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:24:52.611562 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a46bd1-07d3-49d7-86f9-4a303a750675 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5f525dac532047f29df0eb7337b99d54)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:24:52.613664 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a46bd1-07d3-49d7-86f9-4a303a750675 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:24:52.719160 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-92a46bd1-07d3-49d7-86f9-4a303a750675 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5f525dac532047f29df0eb7337b99d54', 'request_id': 'req-92a46bd1-07d3-49d7-86f9-4a303a750675'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:24:52.725056 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-92a46bd1-07d3-49d7-86f9-4a303a750675 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5f525dac532047f29df0eb7337b99d54', 'request_id': 'req-92a46bd1-07d3-49d7-86f9-4a303a750675'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:24:52.800052 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-92a46bd1-07d3-49d7-86f9-4a303a750675 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5f525dac532047f29df0eb7337b99d54', 'request_id': 'req-92a46bd1-07d3-49d7-86f9-4a303a750675'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:24:52.814755 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-92a46bd1-07d3-49d7-86f9-4a303a750675 admin admin] The token cache is being invalidate because project 5f525dac532047f29df0eb7337b99d54 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81750) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 01:24:52.815209 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-92a46bd1-07d3-49d7-86f9-4a303a750675 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:24:52.816085 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-92a46bd1-07d3-49d7-86f9-4a303a750675 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:24:52.822590 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1095/2188] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 01:24:52 2022] DELETE /identity/v3/projects/5f525dac532047f29df0eb7337b99d54 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:24:52.824991 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70386806-0aac-4e5d-87b4-280e145e40ca None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:24:52.824991 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70386806-0aac-4e5d-87b4-280e145e40ca None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:24:52.825576 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70386806-0aac-4e5d-87b4-280e145e40ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:24:52.846257 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-70386806-0aac-4e5d-87b4-280e145e40ca None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:24:52.847632 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-70386806-0aac-4e5d-87b4-280e145e40ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:24:52.869540 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:24:52.919600 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1094/2189] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:24:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:24:52.927689 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135ba169-4f07-4e92-afea-b12ac069f704 None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:24:52.928041 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135ba169-4f07-4e92-afea-b12ac069f704 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:24:52.928407 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135ba169-4f07-4e92-afea-b12ac069f704 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:24:52.948552 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-135ba169-4f07-4e92-afea-b12ac069f704 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:24:52.949799 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-135ba169-4f07-4e92-afea-b12ac069f704 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:24:52.959059 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1096/2190] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:24:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:24:52.963462 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70386806-0aac-4e5d-87b4-280e145e40ca None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:24:52.973432 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70386806-0aac-4e5d-87b4-280e145e40ca None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:24:52.978337 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d78d96c-6af7-44a1-877b-df9548ee9b4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:24:52.979039 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d78d96c-6af7-44a1-877b-df9548ee9b4b admin admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:24:52.979329 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d78d96c-6af7-44a1-877b-df9548ee9b4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:24:52.979609 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d78d96c-6af7-44a1-877b-df9548ee9b4b admin admin] PATH_INFO: `/v3/domains` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:24:52.980637 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d78d96c-6af7-44a1-877b-df9548ee9b4b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:24:52.980637 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d78d96c-6af7-44a1-877b-df9548ee9b4b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:24:52.982286 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d78d96c-6af7-44a1-877b-df9548ee9b4b admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:24:52.994632 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1095/2191] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Aug 31 01:24:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:24:53.000508 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135ba169-4f07-4e92-afea-b12ac069f704 None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:24:53.011335 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135ba169-4f07-4e92-afea-b12ac069f704 None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:24:53.015911 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc92471e-4b9e-4cfd-a8f1-301bc847e4a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:24:53.016509 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc92471e-4b9e-4cfd-a8f1-301bc847e4a1 admin admin] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:24:53.016813 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc92471e-4b9e-4cfd-a8f1-301bc847e4a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:24:53.017086 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc92471e-4b9e-4cfd-a8f1-301bc847e4a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:24:53.017751 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc92471e-4b9e-4cfd-a8f1-301bc847e4a1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:24:53.019642 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc92471e-4b9e-4cfd-a8f1-301bc847e4a1 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:24:53.105124 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1097/2192] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 01:24:52 2022] POST /identity/v3/projects => generated 375 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:24:53.108543 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d78d96c-6af7-44a1-877b-df9548ee9b4b admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:24:53.121915 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d78d96c-6af7-44a1-877b-df9548ee9b4b admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:24:53.193254 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9139e71-60c3-4764-9a7d-027bede0b33a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:24:53.194203 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9139e71-60c3-4764-9a7d-027bede0b33a admin admin] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:24:53.194493 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9139e71-60c3-4764-9a7d-027bede0b33a admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:24:53.194772 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9139e71-60c3-4764-9a7d-027bede0b33a admin admin] PATH_INFO: `/v3/users` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:24:53.195424 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9139e71-60c3-4764-9a7d-027bede0b33a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:24:53.197438 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9139e71-60c3-4764-9a7d-027bede0b33a admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:24:53.290453 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1096/2193] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 01:24:53 2022] POST /identity/v3/users => generated 288 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:24:53.292054 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc92471e-4b9e-4cfd-a8f1-301bc847e4a1 admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:24:53.303390 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc92471e-4b9e-4cfd-a8f1-301bc847e4a1 admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:24:53.308356 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2434363-ce3b-447f-b731-de452d9e9664 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:24:53.308992 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2434363-ce3b-447f-b731-de452d9e9664 admin admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:24:53.309274 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2434363-ce3b-447f-b731-de452d9e9664 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:24:53.309549 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2434363-ce3b-447f-b731-de452d9e9664 admin admin] PATH_INFO: `/v3/roles` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:24:53.310190 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2434363-ce3b-447f-b731-de452d9e9664 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:24:53.311792 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2434363-ce3b-447f-b731-de452d9e9664 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:24:53.323261 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1098/2194] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:24:53 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:24:53.327294 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9139e71-60c3-4764-9a7d-027bede0b33a admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:24:53.338031 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9139e71-60c3-4764-9a7d-027bede0b33a admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:24:53.342639 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2173436b-f70e-46fd-b026-057ae9614db2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:24:53.343305 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2173436b-f70e-46fd-b026-057ae9614db2 admin admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:24:53.343592 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2173436b-f70e-46fd-b026-057ae9614db2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:24:53.343872 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2173436b-f70e-46fd-b026-057ae9614db2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:24:53.344516 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2173436b-f70e-46fd-b026-057ae9614db2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:24:53.346174 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2173436b-f70e-46fd-b026-057ae9614db2 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:24:53.357094 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1097/2195] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:24:53 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:24:53.361147 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2434363-ce3b-447f-b731-de452d9e9664 admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:24:53.372167 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2434363-ce3b-447f-b731-de452d9e9664 admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:24:53.376962 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da526ee9-ab4f-4d16-b892-3c1161270aa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:24:53.377623 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da526ee9-ab4f-4d16-b892-3c1161270aa9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:24:53.377906 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da526ee9-ab4f-4d16-b892-3c1161270aa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:24:53.378215 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da526ee9-ab4f-4d16-b892-3c1161270aa9 admin admin] PATH_INFO: `/v3/projects/6ac413264abe44918f2d1c65a365c063/users/c87f30ec33384ef483ee7bca27d4bb54/roles/d0b9768f4d014766b9326387f2e11f34` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:24:53.394815 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da526ee9-ab4f-4d16-b892-3c1161270aa9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ac413264abe44918f2d1c65a365c063, user_id=c87f30ec33384ef483ee7bca27d4bb54, role_id=d0b9768f4d014766b9326387f2e11f34)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:24:53.397578 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da526ee9-ab4f-4d16-b892-3c1161270aa9 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:24:53.481202 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1099/2196] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 01:24:53 2022] PUT /identity/v3/projects/6ac413264abe44918f2d1c65a365c063/users/c87f30ec33384ef483ee7bca27d4bb54/roles/d0b9768f4d014766b9326387f2e11f34 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 01:24:53.491432 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55188cb0-6297-4e23-b7c6-a02f7ce080ab None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:24:53.491726 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55188cb0-6297-4e23-b7c6-a02f7ce080ab None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:24:53.492035 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55188cb0-6297-4e23-b7c6-a02f7ce080ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:24:53.522092 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-55188cb0-6297-4e23-b7c6-a02f7ce080ab None None] MFA Rules not processed for user `c87f30ec33384ef483ee7bca27d4bb54`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:24:53.523331 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-55188cb0-6297-4e23-b7c6-a02f7ce080ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:24:53.590549 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1098/2197] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:24:53 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:24:53.601544 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da526ee9-ab4f-4d16-b892-3c1161270aa9 admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:24:53.603403 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-da526ee9-ab4f-4d16-b892-3c1161270aa9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:24:53.625540 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:24:53.662029 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da526ee9-ab4f-4d16-b892-3c1161270aa9 admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:24:53.665931 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda1d06b-f6c6-4b9a-aa8b-55d6bf0c66b7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5c8fc69c518412ba1aad61a9f4f558c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:24:53.666549 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda1d06b-f6c6-4b9a-aa8b-55d6bf0c66b7 service cinder] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:24:53.666833 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda1d06b-f6c6-4b9a-aa8b-55d6bf0c66b7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:24:53.667133 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda1d06b-f6c6-4b9a-aa8b-55d6bf0c66b7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:24:53.675686 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda1d06b-f6c6-4b9a-aa8b-55d6bf0c66b7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:24:53.677205 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda1d06b-f6c6-4b9a-aa8b-55d6bf0c66b7 service cinder] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:24:53.686299 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1100/2198] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:24:53 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:24:55.438109 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a9177a-18aa-4c87-b565-829f0ebfb8bc None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:24:55.438613 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a9177a-18aa-4c87-b565-829f0ebfb8bc None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:24:55.438710 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a9177a-18aa-4c87-b565-829f0ebfb8bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:24:55.458469 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-49a9177a-18aa-4c87-b565-829f0ebfb8bc None None] MFA Rules not processed for user `c87f30ec33384ef483ee7bca27d4bb54`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:24:55.459666 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-49a9177a-18aa-4c87-b565-829f0ebfb8bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:24:55.466577 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1099/2199] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:24:55 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:24:59.432451 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d4b951-3ca7-4947-9398-9c431a583202 None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:24:59.432955 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d4b951-3ca7-4947-9398-9c431a583202 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:24:59.433137 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d4b951-3ca7-4947-9398-9c431a583202 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:24:59.452565 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-e9d4b951-3ca7-4947-9398-9c431a583202 None None] MFA Rules not processed for user `b3efb4f9daac4a0ca05a36dd361f17e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:24:59.454045 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-e9d4b951-3ca7-4947-9398-9c431a583202 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:24:59.509387 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1101/2200] 10.222.0.20 () {58 vars in 1009 bytes} [Wed Aug 31 01:24:59 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:24:59.555397 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a9177a-18aa-4c87-b565-829f0ebfb8bc None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:24:59.556914 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-49a9177a-18aa-4c87-b565-829f0ebfb8bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:24:59.570789 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:24:59.628314 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a9177a-18aa-4c87-b565-829f0ebfb8bc None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:24:59.634875 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176966b9-7fde-43f5-beff-cd36adc6b9ef service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc1d010919ec40878ae829cc8dceec4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:24:59.635504 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176966b9-7fde-43f5-beff-cd36adc6b9ef service swift] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:24:59.635824 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176966b9-7fde-43f5-beff-cd36adc6b9ef service swift] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:24:59.636067 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176966b9-7fde-43f5-beff-cd36adc6b9ef service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:24:59.645779 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176966b9-7fde-43f5-beff-cd36adc6b9ef service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:24:59.647360 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176966b9-7fde-43f5-beff-cd36adc6b9ef service swift] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:24:59.655937 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1100/2201] 10.222.0.20 () {60 vars in 1384 bytes} [Wed Aug 31 01:24:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 01:25:00.487859 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d4b951-3ca7-4947-9398-9c431a583202 None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:25:00.490135 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-e9d4b951-3ca7-4947-9398-9c431a583202 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:25:00.595358 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d4b951-3ca7-4947-9398-9c431a583202 None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:25:00.604294 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adcfdb6-b11a-4ced-b4ce-946f0b17f601 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:25:00.605255 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adcfdb6-b11a-4ced-b4ce-946f0b17f601 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:25:00.605697 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adcfdb6-b11a-4ced-b4ce-946f0b17f601 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:25:00.606125 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adcfdb6-b11a-4ced-b4ce-946f0b17f601 admin admin] PATH_INFO: `/v3/users/8a3fb0c04c874d37b66916032680556b` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:25:00.607735 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2adcfdb6-b11a-4ced-b4ce-946f0b17f601 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a3fb0c04c874d37b66916032680556b)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:25:00.610081 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2adcfdb6-b11a-4ced-b4ce-946f0b17f601 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:25:01.150646 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-2adcfdb6-b11a-4ced-b4ce-946f0b17f601 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a3fb0c04c874d37b66916032680556b'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:25:01.184760 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-2adcfdb6-b11a-4ced-b4ce-946f0b17f601 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a3fb0c04c874d37b66916032680556b'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:25:01.184760 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-2adcfdb6-b11a-4ced-b4ce-946f0b17f601 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a3fb0c04c874d37b66916032680556b'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:25:01.192326 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:25:01.197963 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-2adcfdb6-b11a-4ced-b4ce-946f0b17f601 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a3fb0c04c874d37b66916032680556b'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:25:01.278873 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-2adcfdb6-b11a-4ced-b4ce-946f0b17f601 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a3fb0c04c874d37b66916032680556b'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:25:01.290915 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1102/2202] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 01:25:00 2022] DELETE /identity/v3/users/8a3fb0c04c874d37b66916032680556b => generated 0 bytes in 804 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:25:01.296390 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494e16db-acb8-432c-b25a-ecd101fc307c None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:25:01.297439 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494e16db-acb8-432c-b25a-ecd101fc307c None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:25:01.297887 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494e16db-acb8-432c-b25a-ecd101fc307c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:25:01.359642 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-494e16db-acb8-432c-b25a-ecd101fc307c None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:25:01.362250 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-494e16db-acb8-432c-b25a-ecd101fc307c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:25:01.462780 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1101/2203] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:25:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:25:01.475341 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adcfdb6-b11a-4ced-b4ce-946f0b17f601 admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:25:01.478062 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-2adcfdb6-b11a-4ced-b4ce-946f0b17f601 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:25:01.579344 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adcfdb6-b11a-4ced-b4ce-946f0b17f601 admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:25:01.583775 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ad86ed-40fe-46b3-9152-4e883ccbc121 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0817c3320d5d41d8bba0550c8ae8dd60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:25:01.584409 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ad86ed-40fe-46b3-9152-4e883ccbc121 service neutron] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:25:01.584689 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ad86ed-40fe-46b3-9152-4e883ccbc121 service neutron] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:25:01.585007 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ad86ed-40fe-46b3-9152-4e883ccbc121 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:25:01.603705 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ad86ed-40fe-46b3-9152-4e883ccbc121 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:25:01.603705 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ad86ed-40fe-46b3-9152-4e883ccbc121 service neutron] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:25:01.616370 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1103/2204] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:25:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:25:02.279027 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494e16db-acb8-432c-b25a-ecd101fc307c None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:25:02.280863 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-494e16db-acb8-432c-b25a-ecd101fc307c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:25:02.298713 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494e16db-acb8-432c-b25a-ecd101fc307c None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:25:02.303950 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264b9398-456c-4b90-a477-217a9fad0314 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:25:02.304928 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264b9398-456c-4b90-a477-217a9fad0314 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:25:02.305449 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264b9398-456c-4b90-a477-217a9fad0314 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:25:02.305941 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264b9398-456c-4b90-a477-217a9fad0314 admin admin] PATH_INFO: `/v3/projects/f93acac1ed504852b8239acc71779630` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:25:02.307462 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264b9398-456c-4b90-a477-217a9fad0314 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f93acac1ed504852b8239acc71779630)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:25:02.309730 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264b9398-456c-4b90-a477-217a9fad0314 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:25:02.523287 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-264b9398-456c-4b90-a477-217a9fad0314 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f93acac1ed504852b8239acc71779630', 'request_id': 'req-264b9398-456c-4b90-a477-217a9fad0314'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:25:02.541468 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-264b9398-456c-4b90-a477-217a9fad0314 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f93acac1ed504852b8239acc71779630', 'request_id': 'req-264b9398-456c-4b90-a477-217a9fad0314'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:25:02.555504 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-264b9398-456c-4b90-a477-217a9fad0314 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f93acac1ed504852b8239acc71779630', 'request_id': 'req-264b9398-456c-4b90-a477-217a9fad0314'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:25:02.668584 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-264b9398-456c-4b90-a477-217a9fad0314 admin admin] The token cache is being invalidate because project f93acac1ed504852b8239acc71779630 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81749) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 01:25:02.669929 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-264b9398-456c-4b90-a477-217a9fad0314 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:25:02.670848 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-264b9398-456c-4b90-a477-217a9fad0314 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:25:02.672506 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1102/2205] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 01:25:02 2022] DELETE /identity/v3/projects/f93acac1ed504852b8239acc71779630 => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:25:02.678218 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d943b25-d311-4ba4-b732-ca1c22889a66 None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:25:02.678772 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d943b25-d311-4ba4-b732-ca1c22889a66 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:25:02.678772 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d943b25-d311-4ba4-b732-ca1c22889a66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:25:02.723284 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-4d943b25-d311-4ba4-b732-ca1c22889a66 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:25:02.739654 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-4d943b25-d311-4ba4-b732-ca1c22889a66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:25:02.838654 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1104/2206] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:25:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:25:02.848365 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c772fc-0a07-4511-af17-977f224c94ac None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:25:02.848683 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c772fc-0a07-4511-af17-977f224c94ac None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:25:02.849007 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c772fc-0a07-4511-af17-977f224c94ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:25:02.874287 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-82c772fc-0a07-4511-af17-977f224c94ac None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:25:02.875595 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-82c772fc-0a07-4511-af17-977f224c94ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:25:02.884246 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1103/2207] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:25:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:25:02.888660 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d943b25-d311-4ba4-b732-ca1c22889a66 None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:25:02.903685 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d943b25-d311-4ba4-b732-ca1c22889a66 None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:25:02.915401 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bffa43-16a4-48b3-85b4-c3d70c2a924c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:25:02.915897 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bffa43-16a4-48b3-85b4-c3d70c2a924c admin admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:25:02.916182 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bffa43-16a4-48b3-85b4-c3d70c2a924c admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:25:02.916461 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bffa43-16a4-48b3-85b4-c3d70c2a924c admin admin] PATH_INFO: `/v3/domains` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:25:02.917554 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bffa43-16a4-48b3-85b4-c3d70c2a924c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:25:02.917924 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bffa43-16a4-48b3-85b4-c3d70c2a924c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:25:02.920031 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bffa43-16a4-48b3-85b4-c3d70c2a924c admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:25:02.932906 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1105/2208] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Aug 31 01:25:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 01:25:02.937921 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c772fc-0a07-4511-af17-977f224c94ac None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:25:02.947861 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c772fc-0a07-4511-af17-977f224c94ac None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:25:02.953368 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6488d75e-415b-44ea-a19f-e256a5005c52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:25:02.954325 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6488d75e-415b-44ea-a19f-e256a5005c52 admin admin] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:25:02.954602 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6488d75e-415b-44ea-a19f-e256a5005c52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:25:02.954870 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6488d75e-415b-44ea-a19f-e256a5005c52 admin admin] PATH_INFO: `/v3/projects` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:25:02.955488 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6488d75e-415b-44ea-a19f-e256a5005c52 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:25:02.958141 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6488d75e-415b-44ea-a19f-e256a5005c52 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:25:02.976430 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:25:03.422910 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1104/2209] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 01:25:02 2022] POST /identity/v3/projects => generated 365 bytes in 485 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:25:03.426891 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bffa43-16a4-48b3-85b4-c3d70c2a924c admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:25:03.437127 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bffa43-16a4-48b3-85b4-c3d70c2a924c admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:25:03.497124 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea42eaf-a303-4e92-8229-c317fd02221c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:25:03.497743 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea42eaf-a303-4e92-8229-c317fd02221c admin admin] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:25:03.498027 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea42eaf-a303-4e92-8229-c317fd02221c admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:25:03.498308 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea42eaf-a303-4e92-8229-c317fd02221c admin admin] PATH_INFO: `/v3/users` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:25:03.499014 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea42eaf-a303-4e92-8229-c317fd02221c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:25:03.501008 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea42eaf-a303-4e92-8229-c317fd02221c admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:25:03.600584 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1106/2210] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 01:25:03 2022] POST /identity/v3/users => generated 283 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:25:03.604400 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6488d75e-415b-44ea-a19f-e256a5005c52 admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:25:03.619707 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6488d75e-415b-44ea-a19f-e256a5005c52 admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:25:03.625705 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0879f8a-827c-4ff4-9cce-43784f2b1256 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:25:03.626619 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0879f8a-827c-4ff4-9cce-43784f2b1256 admin admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:25:03.627169 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0879f8a-827c-4ff4-9cce-43784f2b1256 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:25:03.627675 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0879f8a-827c-4ff4-9cce-43784f2b1256 admin admin] PATH_INFO: `/v3/roles` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:25:03.629751 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0879f8a-827c-4ff4-9cce-43784f2b1256 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:25:03.632104 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0879f8a-827c-4ff4-9cce-43784f2b1256 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:25:03.643321 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1105/2211] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:25:03 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:25:03.650995 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea42eaf-a303-4e92-8229-c317fd02221c admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:25:03.664589 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea42eaf-a303-4e92-8229-c317fd02221c admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:25:03.670114 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e11cc11-8648-41f0-ad59-925906ad887f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:25:03.670752 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e11cc11-8648-41f0-ad59-925906ad887f admin admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:25:03.671043 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e11cc11-8648-41f0-ad59-925906ad887f admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:25:03.671320 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e11cc11-8648-41f0-ad59-925906ad887f admin admin] PATH_INFO: `/v3/roles` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:25:03.672136 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e11cc11-8648-41f0-ad59-925906ad887f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:25:03.673776 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e11cc11-8648-41f0-ad59-925906ad887f admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:25:03.675926 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:25:03.704809 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1107/2212] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:25:03 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 01:25:03.706076 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0879f8a-827c-4ff4-9cce-43784f2b1256 admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:25:03.723586 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0879f8a-827c-4ff4-9cce-43784f2b1256 admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:25:03.728390 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7da4f8c-c13a-463f-8a69-d9211eed678d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:25:03.729082 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7da4f8c-c13a-463f-8a69-d9211eed678d admin admin] REQUEST_METHOD: `PUT` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:25:03.729481 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7da4f8c-c13a-463f-8a69-d9211eed678d admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:25:03.729760 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7da4f8c-c13a-463f-8a69-d9211eed678d admin admin] PATH_INFO: `/v3/projects/99b04f41cebb4755aa0783764c1299aa/users/8b3e4ad415b949648bd38b103de5735f/roles/d0b9768f4d014766b9326387f2e11f34` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:25:03.747177 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7da4f8c-c13a-463f-8a69-d9211eed678d admin admin] RBAC: Authorizing `identity:create_grant(project_id=99b04f41cebb4755aa0783764c1299aa, user_id=8b3e4ad415b949648bd38b103de5735f, role_id=d0b9768f4d014766b9326387f2e11f34)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:25:03.750058 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7da4f8c-c13a-463f-8a69-d9211eed678d admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:25:03.825506 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1106/2213] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 01:25:03 2022] PUT /identity/v3/projects/99b04f41cebb4755aa0783764c1299aa/users/8b3e4ad415b949648bd38b103de5735f/roles/d0b9768f4d014766b9326387f2e11f34 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 01:25:03.836321 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c82eed2-1e7e-4131-9cb6-d3f8fd353335 None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:25:03.836688 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c82eed2-1e7e-4131-9cb6-d3f8fd353335 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:25:03.836908 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c82eed2-1e7e-4131-9cb6-d3f8fd353335 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:25:03.869957 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-0c82eed2-1e7e-4131-9cb6-d3f8fd353335 None None] MFA Rules not processed for user `8b3e4ad415b949648bd38b103de5735f`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:25:03.871295 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-0c82eed2-1e7e-4131-9cb6-d3f8fd353335 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:25:03.962071 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1108/2214] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:25:03 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:25:03.989815 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7da4f8c-c13a-463f-8a69-d9211eed678d admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:25:03.992070 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-f7da4f8c-c13a-463f-8a69-d9211eed678d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:25:04.059939 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7da4f8c-c13a-463f-8a69-d9211eed678d admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:25:04.064098 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b1df7d-8fb9-4b22-8bd1-1bc136257b9b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5c8fc69c518412ba1aad61a9f4f558c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:25:04.065059 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b1df7d-8fb9-4b22-8bd1-1bc136257b9b service cinder] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:25:04.065350 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b1df7d-8fb9-4b22-8bd1-1bc136257b9b service cinder] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:25:04.065624 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b1df7d-8fb9-4b22-8bd1-1bc136257b9b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:25:04.075978 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b1df7d-8fb9-4b22-8bd1-1bc136257b9b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:25:04.077993 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b1df7d-8fb9-4b22-8bd1-1bc136257b9b service cinder] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:25:04.088299 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1107/2215] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:25:03 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:25:13.950763 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=34 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:25:14.084959 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=43,keystone:INSERT=1 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:25:19.790299 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da09d6a9-b5ce-47bc-aae2-7488b3273b3d None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:25:19.790777 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da09d6a9-b5ce-47bc-aae2-7488b3273b3d None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:25:19.790920 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da09d6a9-b5ce-47bc-aae2-7488b3273b3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:25:19.812044 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-da09d6a9-b5ce-47bc-aae2-7488b3273b3d None None] MFA Rules not processed for user `b3efb4f9daac4a0ca05a36dd361f17e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:25:19.813329 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-da09d6a9-b5ce-47bc-aae2-7488b3273b3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:25:19.871372 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1109/2216] 10.222.0.20 () {58 vars in 1009 bytes} [Wed Aug 31 01:25:19 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:25:19.883581 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b1df7d-8fb9-4b22-8bd1-1bc136257b9b service cinder] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:25:19.885099 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-f6b1df7d-8fb9-4b22-8bd1-1bc136257b9b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:25:19.965108 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b1df7d-8fb9-4b22-8bd1-1bc136257b9b service cinder] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:25:19.970366 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36da7aba-9ed5-45fa-89e4-3a207b0fc370 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc1d010919ec40878ae829cc8dceec4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:25:19.970990 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36da7aba-9ed5-45fa-89e4-3a207b0fc370 service swift] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:25:19.971278 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36da7aba-9ed5-45fa-89e4-3a207b0fc370 service swift] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:25:19.971580 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36da7aba-9ed5-45fa-89e4-3a207b0fc370 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:25:19.982643 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36da7aba-9ed5-45fa-89e4-3a207b0fc370 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:25:19.984292 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36da7aba-9ed5-45fa-89e4-3a207b0fc370 service swift] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:25:19.994869 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1108/2217] 10.222.0.20 () {60 vars in 1384 bytes} [Wed Aug 31 01:25:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 01:25:20.270992 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46937c6-3739-4a46-96cb-52112d9465e3 None None] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:25:20.271283 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46937c6-3739-4a46-96cb-52112d9465e3 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:25:20.271894 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46937c6-3739-4a46-96cb-52112d9465e3 None None] PATH_INFO: `/` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:25:20.273072 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1110/2218] 10.222.0.20 () {54 vars in 921 bytes} [Wed Aug 31 01:25:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:25:20.279067 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a391431-0936-4763-8149-951164bc994f None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:25:20.279432 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a391431-0936-4763-8149-951164bc994f None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:25:20.279741 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a391431-0936-4763-8149-951164bc994f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:25:20.300011 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-6a391431-0936-4763-8149-951164bc994f None None] MFA Rules not processed for user `8744c9de803048f583d0317d01f6a540`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:25:20.301285 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-6a391431-0936-4763-8149-951164bc994f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:25:20.372948 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1109/2219] 10.222.0.20 () {58 vars in 1018 bytes} [Wed Aug 31 01:25:20 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:25:20.384376 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46937c6-3739-4a46-96cb-52112d9465e3 None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:25:20.386372 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-e46937c6-3739-4a46-96cb-52112d9465e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:25:20.459339 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46937c6-3739-4a46-96cb-52112d9465e3 None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:25:20.463082 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605f6262-c5ca-40cd-b2dd-7bc3fa8537b7 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8590c288028418ebe7a4163fd8039f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:25:20.463726 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605f6262-c5ca-40cd-b2dd-7bc3fa8537b7 service glance] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:25:20.463988 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605f6262-c5ca-40cd-b2dd-7bc3fa8537b7 service glance] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:25:20.464294 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605f6262-c5ca-40cd-b2dd-7bc3fa8537b7 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:25:20.474703 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605f6262-c5ca-40cd-b2dd-7bc3fa8537b7 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:25:20.476129 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605f6262-c5ca-40cd-b2dd-7bc3fa8537b7 service glance] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:25:20.486866 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1111/2220] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:25:20 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:25:20.789352 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fb47b7-7a59-4e48-ae79-06e1db753da6 None None] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:25:20.789740 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fb47b7-7a59-4e48-ae79-06e1db753da6 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:25:20.791913 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fb47b7-7a59-4e48-ae79-06e1db753da6 None None] PATH_INFO: `/` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:25:20.791913 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1110/2221] 10.222.0.20 () {54 vars in 921 bytes} [Wed Aug 31 01:25:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:25:20.796877 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4564ba6d-74b7-4ebf-ac42-4b465dd51e1c None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:25:20.797385 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4564ba6d-74b7-4ebf-ac42-4b465dd51e1c None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:25:20.797690 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4564ba6d-74b7-4ebf-ac42-4b465dd51e1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:25:20.820021 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-4564ba6d-74b7-4ebf-ac42-4b465dd51e1c None None] MFA Rules not processed for user `0817c3320d5d41d8bba0550c8ae8dd60`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:25:20.821275 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-4564ba6d-74b7-4ebf-ac42-4b465dd51e1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:25:20.877229 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1112/2222] 10.222.0.20 () {58 vars in 1018 bytes} [Wed Aug 31 01:25:20 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:25:20.890098 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fb47b7-7a59-4e48-ae79-06e1db753da6 None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:25:20.891844 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-10fb47b7-7a59-4e48-ae79-06e1db753da6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:25:20.908621 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fb47b7-7a59-4e48-ae79-06e1db753da6 None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:25:20.912845 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d0ef16-4f90-4bd5-b99a-45a58db28840 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0817c3320d5d41d8bba0550c8ae8dd60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:25:20.913598 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d0ef16-4f90-4bd5-b99a-45a58db28840 service neutron] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:25:20.913876 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d0ef16-4f90-4bd5-b99a-45a58db28840 service neutron] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:25:20.914279 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d0ef16-4f90-4bd5-b99a-45a58db28840 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:25:20.921790 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d0ef16-4f90-4bd5-b99a-45a58db28840 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:25:20.923433 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d0ef16-4f90-4bd5-b99a-45a58db28840 service neutron] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:25:20.932227 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1111/2223] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:25:20 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:25:21.019035 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ad32be-8e20-4819-8d9b-2ca369e5ba73 None None] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:25:21.019333 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ad32be-8e20-4819-8d9b-2ca369e5ba73 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:25:21.019614 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ad32be-8e20-4819-8d9b-2ca369e5ba73 None None] PATH_INFO: `/` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:25:21.021022 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1113/2224] 10.222.0.20 () {54 vars in 921 bytes} [Wed Aug 31 01:25:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:25:21.027442 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc6fb2a-17dd-4b13-99c5-3c7d819959dc None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:25:21.027988 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc6fb2a-17dd-4b13-99c5-3c7d819959dc None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:25:21.028335 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc6fb2a-17dd-4b13-99c5-3c7d819959dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:25:21.048329 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-7dc6fb2a-17dd-4b13-99c5-3c7d819959dc None None] MFA Rules not processed for user `8744c9de803048f583d0317d01f6a540`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:25:21.054229 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-7dc6fb2a-17dd-4b13-99c5-3c7d819959dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:25:21.061894 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1112/2225] 10.222.0.20 () {58 vars in 1018 bytes} [Wed Aug 31 01:25:21 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:25:21.079242 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ad32be-8e20-4819-8d9b-2ca369e5ba73 None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:25:21.088785 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e1475e-171a-46c1-8425-a8ca78fa6950 None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:25:21.089548 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e1475e-171a-46c1-8425-a8ca78fa6950 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:25:21.090014 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e1475e-171a-46c1-8425-a8ca78fa6950 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:25:21.092416 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ad32be-8e20-4819-8d9b-2ca369e5ba73 None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:25:21.096476 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11935d6-4aaa-418b-8b39-4bf51257fb38 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0817c3320d5d41d8bba0550c8ae8dd60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:25:21.097221 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11935d6-4aaa-418b-8b39-4bf51257fb38 service neutron] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:25:21.097659 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11935d6-4aaa-418b-8b39-4bf51257fb38 service neutron] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:25:21.098125 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11935d6-4aaa-418b-8b39-4bf51257fb38 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:25:21.110718 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11935d6-4aaa-418b-8b39-4bf51257fb38 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:25:21.112245 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11935d6-4aaa-418b-8b39-4bf51257fb38 service neutron] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:25:21.112985 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-80e1475e-171a-46c1-8425-a8ca78fa6950 None None] MFA Rules not processed for user `b3efb4f9daac4a0ca05a36dd361f17e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:25:21.114781 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-80e1475e-171a-46c1-8425-a8ca78fa6950 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:25:21.122912 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1113/2226] 10.222.0.20 () {58 vars in 967 bytes} [Wed Aug 31 01:25:21 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 01:25:21.125826 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1114/2227] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:25:21 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:25:21.141283 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e1475e-171a-46c1-8425-a8ca78fa6950 None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:25:21.154093 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e1475e-171a-46c1-8425-a8ca78fa6950 None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:25:21.159758 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861cff5e-239a-42d6-859c-83a4158719dc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc1d010919ec40878ae829cc8dceec4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:25:21.160321 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861cff5e-239a-42d6-859c-83a4158719dc service swift] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:25:21.160606 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861cff5e-239a-42d6-859c-83a4158719dc service swift] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:25:21.160915 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861cff5e-239a-42d6-859c-83a4158719dc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:25:21.171318 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861cff5e-239a-42d6-859c-83a4158719dc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:25:21.172978 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861cff5e-239a-42d6-859c-83a4158719dc service swift] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:25:21.182232 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1114/2228] 10.222.0.20 () {60 vars in 1384 bytes} [Wed Aug 31 01:25:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 01:25:22.097963 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4315ae8a-1f6c-4377-a7f7-d7e79f7d7a9d None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:25:22.098851 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4315ae8a-1f6c-4377-a7f7-d7e79f7d7a9d None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:25:22.099136 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4315ae8a-1f6c-4377-a7f7-d7e79f7d7a9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:25:22.118454 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-4315ae8a-1f6c-4377-a7f7-d7e79f7d7a9d None None] MFA Rules not processed for user `b3efb4f9daac4a0ca05a36dd361f17e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:25:22.120045 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-4315ae8a-1f6c-4377-a7f7-d7e79f7d7a9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:25:22.127344 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1115/2229] 10.222.0.20 () {58 vars in 1009 bytes} [Wed Aug 31 01:25:22 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 01:25:22.142907 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861cff5e-239a-42d6-859c-83a4158719dc service swift] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:25:22.150727 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861cff5e-239a-42d6-859c-83a4158719dc service swift] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:25:22.155713 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7408a129-e487-496d-8fe4-94ef5a6d1c68 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc1d010919ec40878ae829cc8dceec4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:25:22.156299 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7408a129-e487-496d-8fe4-94ef5a6d1c68 service swift] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:25:22.156575 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7408a129-e487-496d-8fe4-94ef5a6d1c68 service swift] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:25:22.156878 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7408a129-e487-496d-8fe4-94ef5a6d1c68 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:25:22.165049 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7408a129-e487-496d-8fe4-94ef5a6d1c68 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:25:22.166660 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7408a129-e487-496d-8fe4-94ef5a6d1c68 service swift] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:25:22.175079 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1115/2230] 10.222.0.20 () {60 vars in 1384 bytes} [Wed Aug 31 01:25:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 01:25:22.874706 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17919b31-ad2b-4eca-963f-b68fb9888398 None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:25:22.884192 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17919b31-ad2b-4eca-963f-b68fb9888398 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:25:22.884192 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17919b31-ad2b-4eca-963f-b68fb9888398 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:25:22.897010 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-17919b31-ad2b-4eca-963f-b68fb9888398 None None] MFA Rules not processed for user `b3efb4f9daac4a0ca05a36dd361f17e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:25:22.898227 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-17919b31-ad2b-4eca-963f-b68fb9888398 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:25:22.912448 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1116/2231] 10.222.0.20 () {58 vars in 967 bytes} [Wed Aug 31 01:25:22 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:25:22.924851 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7408a129-e487-496d-8fe4-94ef5a6d1c68 service swift] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:25:22.936888 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7408a129-e487-496d-8fe4-94ef5a6d1c68 service swift] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:25:22.942892 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f277e7-092c-4522-98ee-be7096e17a05 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc1d010919ec40878ae829cc8dceec4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:25:22.943510 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f277e7-092c-4522-98ee-be7096e17a05 service swift] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:25:22.943802 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f277e7-092c-4522-98ee-be7096e17a05 service swift] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:25:22.944081 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f277e7-092c-4522-98ee-be7096e17a05 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:25:22.953325 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f277e7-092c-4522-98ee-be7096e17a05 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:25:22.955163 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f277e7-092c-4522-98ee-be7096e17a05 service swift] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:25:22.964345 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1116/2232] 10.222.0.20 () {60 vars in 1384 bytes} [Wed Aug 31 01:25:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 01:25:25.697909 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17919b31-ad2b-4eca-963f-b68fb9888398 None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:25:25.699319 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-17919b31-ad2b-4eca-963f-b68fb9888398 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:25:25.733169 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:25:25.764512 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17919b31-ad2b-4eca-963f-b68fb9888398 None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:25:25.769895 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4e4f52-0763-4a0b-8510-85df38af9ca0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:25:25.770528 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4e4f52-0763-4a0b-8510-85df38af9ca0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:25:25.770813 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4e4f52-0763-4a0b-8510-85df38af9ca0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:25:25.771303 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4e4f52-0763-4a0b-8510-85df38af9ca0 admin admin] PATH_INFO: `/v3/users/c87f30ec33384ef483ee7bca27d4bb54` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:25:25.772367 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4e4f52-0763-4a0b-8510-85df38af9ca0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c87f30ec33384ef483ee7bca27d4bb54)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:25:25.774378 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4e4f52-0763-4a0b-8510-85df38af9ca0 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:25:25.988772 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-ac4e4f52-0763-4a0b-8510-85df38af9ca0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c87f30ec33384ef483ee7bca27d4bb54'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:25:26.000517 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-ac4e4f52-0763-4a0b-8510-85df38af9ca0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c87f30ec33384ef483ee7bca27d4bb54'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:25:26.001222 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-ac4e4f52-0763-4a0b-8510-85df38af9ca0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c87f30ec33384ef483ee7bca27d4bb54'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:25:26.008940 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-ac4e4f52-0763-4a0b-8510-85df38af9ca0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c87f30ec33384ef483ee7bca27d4bb54'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:25:26.079446 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-ac4e4f52-0763-4a0b-8510-85df38af9ca0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c87f30ec33384ef483ee7bca27d4bb54'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:25:26.085943 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1117/2233] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 01:25:25 2022] DELETE /identity/v3/users/c87f30ec33384ef483ee7bca27d4bb54 => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:25:26.090437 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea132984-84b1-4039-b0c6-0e39cba842ac None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:25:26.091133 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea132984-84b1-4039-b0c6-0e39cba842ac None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:25:26.091133 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea132984-84b1-4039-b0c6-0e39cba842ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:25:26.109959 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-ea132984-84b1-4039-b0c6-0e39cba842ac None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:25:26.111181 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-ea132984-84b1-4039-b0c6-0e39cba842ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:25:26.121028 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:25:26.172537 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1117/2234] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:25:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:25:26.182570 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4e4f52-0763-4a0b-8510-85df38af9ca0 admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:25:26.184001 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-ac4e4f52-0763-4a0b-8510-85df38af9ca0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:25:26.244776 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4e4f52-0763-4a0b-8510-85df38af9ca0 admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:25:26.248431 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d40fa3-4b48-4c53-ac67-0941b7b97545 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0817c3320d5d41d8bba0550c8ae8dd60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:25:26.249097 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d40fa3-4b48-4c53-ac67-0941b7b97545 service neutron] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:25:26.249385 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d40fa3-4b48-4c53-ac67-0941b7b97545 service neutron] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:25:26.249664 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d40fa3-4b48-4c53-ac67-0941b7b97545 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:25:26.265793 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d40fa3-4b48-4c53-ac67-0941b7b97545 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:25:26.267259 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d40fa3-4b48-4c53-ac67-0941b7b97545 service neutron] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:25:26.277493 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1118/2235] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:25:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:25:26.543630 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea132984-84b1-4039-b0c6-0e39cba842ac None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:25:26.545200 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-ea132984-84b1-4039-b0c6-0e39cba842ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:25:26.561240 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea132984-84b1-4039-b0c6-0e39cba842ac None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:25:26.566249 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fd31f0-0498-4962-9d2a-ab800f5434ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:25:26.566914 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fd31f0-0498-4962-9d2a-ab800f5434ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:25:26.567205 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fd31f0-0498-4962-9d2a-ab800f5434ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:25:26.567523 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fd31f0-0498-4962-9d2a-ab800f5434ac admin admin] PATH_INFO: `/v3/projects/6ac413264abe44918f2d1c65a365c063` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:25:26.568892 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fd31f0-0498-4962-9d2a-ab800f5434ac admin admin] RBAC: Authorizing `identity:delete_project(project_id=6ac413264abe44918f2d1c65a365c063)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:25:26.570853 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fd31f0-0498-4962-9d2a-ab800f5434ac admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:25:26.714002 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-61fd31f0-0498-4962-9d2a-ab800f5434ac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6ac413264abe44918f2d1c65a365c063', 'request_id': 'req-61fd31f0-0498-4962-9d2a-ab800f5434ac'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:25:26.723618 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-61fd31f0-0498-4962-9d2a-ab800f5434ac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6ac413264abe44918f2d1c65a365c063', 'request_id': 'req-61fd31f0-0498-4962-9d2a-ab800f5434ac'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:25:26.730026 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-61fd31f0-0498-4962-9d2a-ab800f5434ac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6ac413264abe44918f2d1c65a365c063', 'request_id': 'req-61fd31f0-0498-4962-9d2a-ab800f5434ac'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:25:26.821342 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-61fd31f0-0498-4962-9d2a-ab800f5434ac admin admin] The token cache is being invalidate because project 6ac413264abe44918f2d1c65a365c063 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81749) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 01:25:26.821631 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-61fd31f0-0498-4962-9d2a-ab800f5434ac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:25:26.822263 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-61fd31f0-0498-4962-9d2a-ab800f5434ac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:25:26.823764 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1118/2236] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 01:25:26 2022] DELETE /identity/v3/projects/6ac413264abe44918f2d1c65a365c063 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:25:32.559216 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ceeb57-6c68-41bb-a1ce-23e6ea953dc0 None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:25:32.559216 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ceeb57-6c68-41bb-a1ce-23e6ea953dc0 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:25:32.559216 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ceeb57-6c68-41bb-a1ce-23e6ea953dc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:25:32.579515 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-42ceeb57-6c68-41bb-a1ce-23e6ea953dc0 None None] MFA Rules not processed for user `b3efb4f9daac4a0ca05a36dd361f17e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:25:32.580776 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-42ceeb57-6c68-41bb-a1ce-23e6ea953dc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:25:32.627547 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:25:32.636668 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1119/2237] 10.222.0.20 () {58 vars in 1009 bytes} [Wed Aug 31 01:25:32 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:25:32.649747 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fd31f0-0498-4962-9d2a-ab800f5434ac admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:25:32.651203 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-61fd31f0-0498-4962-9d2a-ab800f5434ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:25:32.728228 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fd31f0-0498-4962-9d2a-ab800f5434ac admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:25:32.733344 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adfaad5-f64c-4671-96ee-fdb665573d52 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc1d010919ec40878ae829cc8dceec4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:25:32.733896 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adfaad5-f64c-4671-96ee-fdb665573d52 service swift] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:25:32.734183 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adfaad5-f64c-4671-96ee-fdb665573d52 service swift] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:25:32.734491 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adfaad5-f64c-4671-96ee-fdb665573d52 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:25:32.743375 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adfaad5-f64c-4671-96ee-fdb665573d52 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:25:32.746358 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adfaad5-f64c-4671-96ee-fdb665573d52 service swift] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:25:32.753222 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1119/2238] 10.222.0.20 () {60 vars in 1384 bytes} [Wed Aug 31 01:25:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 01:25:42.718690 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=1 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:25:42.750063 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=79,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:25:46.253698 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ceeb57-6c68-41bb-a1ce-23e6ea953dc0 None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:25:46.253698 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-42ceeb57-6c68-41bb-a1ce-23e6ea953dc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:25:46.339798 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ceeb57-6c68-41bb-a1ce-23e6ea953dc0 None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:25:46.345077 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b6b16b-46ac-4970-a96d-657f50e215f6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dea0fe5b1d6b405cb47b95192580303b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:25:46.345751 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b6b16b-46ac-4970-a96d-657f50e215f6 service placement] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:25:46.346034 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b6b16b-46ac-4970-a96d-657f50e215f6 service placement] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:25:46.346313 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b6b16b-46ac-4970-a96d-657f50e215f6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:25:46.348079 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-e5b6b16b-46ac-4970-a96d-657f50e215f6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:25:46.359851 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b6b16b-46ac-4970-a96d-657f50e215f6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:25:46.361510 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b6b16b-46ac-4970-a96d-657f50e215f6 service placement] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:25:46.372416 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1120/2239] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:25:46 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:25:47.502833 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adfaad5-f64c-4671-96ee-fdb665573d52 service swift] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:25:47.504555 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-9adfaad5-f64c-4671-96ee-fdb665573d52 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:25:47.603059 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adfaad5-f64c-4671-96ee-fdb665573d52 service swift] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:25:47.609746 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bed882-bcf9-47d8-837f-2714527dba40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:25:47.610613 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bed882-bcf9-47d8-837f-2714527dba40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:25:47.611054 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bed882-bcf9-47d8-837f-2714527dba40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:25:47.611505 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bed882-bcf9-47d8-837f-2714527dba40 admin admin] PATH_INFO: `/v3/users/8b3e4ad415b949648bd38b103de5735f` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:25:47.612962 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bed882-bcf9-47d8-837f-2714527dba40 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8b3e4ad415b949648bd38b103de5735f)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:25:47.615082 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bed882-bcf9-47d8-837f-2714527dba40 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:25:47.803771 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-28bed882-bcf9-47d8-837f-2714527dba40 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8b3e4ad415b949648bd38b103de5735f'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:25:47.850136 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-28bed882-bcf9-47d8-837f-2714527dba40 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8b3e4ad415b949648bd38b103de5735f'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:25:47.850983 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-28bed882-bcf9-47d8-837f-2714527dba40 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8b3e4ad415b949648bd38b103de5735f'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:25:47.860176 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-28bed882-bcf9-47d8-837f-2714527dba40 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8b3e4ad415b949648bd38b103de5735f'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:25:47.861034 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-28bed882-bcf9-47d8-837f-2714527dba40 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8b3e4ad415b949648bd38b103de5735f'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:25:47.888919 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1120/2240] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 01:25:47 2022] DELETE /identity/v3/users/8b3e4ad415b949648bd38b103de5735f => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:25:47.893282 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51410a3b-52ed-4093-8ab2-80bce5863f07 None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:25:47.893856 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51410a3b-52ed-4093-8ab2-80bce5863f07 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:25:47.894062 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51410a3b-52ed-4093-8ab2-80bce5863f07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:25:47.924020 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-51410a3b-52ed-4093-8ab2-80bce5863f07 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:25:47.925380 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-51410a3b-52ed-4093-8ab2-80bce5863f07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:25:47.995636 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1121/2241] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:25:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:25:48.007339 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bed882-bcf9-47d8-837f-2714527dba40 admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:25:48.010193 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-28bed882-bcf9-47d8-837f-2714527dba40 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:25:48.100977 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bed882-bcf9-47d8-837f-2714527dba40 admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:25:48.112957 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a89ad6-aaac-4180-a248-21cf908e6f3c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0817c3320d5d41d8bba0550c8ae8dd60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:25:48.112957 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a89ad6-aaac-4180-a248-21cf908e6f3c service neutron] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:25:48.112957 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a89ad6-aaac-4180-a248-21cf908e6f3c service neutron] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:25:48.113582 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a89ad6-aaac-4180-a248-21cf908e6f3c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:25:48.126448 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a89ad6-aaac-4180-a248-21cf908e6f3c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:25:48.127994 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a89ad6-aaac-4180-a248-21cf908e6f3c service neutron] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:25:48.142397 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1121/2242] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:25:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:25:48.681165 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51410a3b-52ed-4093-8ab2-80bce5863f07 None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:25:48.682619 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-51410a3b-52ed-4093-8ab2-80bce5863f07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:25:48.694766 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51410a3b-52ed-4093-8ab2-80bce5863f07 None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:25:48.699878 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6f9a37-b235-4eda-b0d3-30ca2d54c16f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:25:48.700519 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6f9a37-b235-4eda-b0d3-30ca2d54c16f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:25:48.700832 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6f9a37-b235-4eda-b0d3-30ca2d54c16f admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:25:48.701115 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6f9a37-b235-4eda-b0d3-30ca2d54c16f admin admin] PATH_INFO: `/v3/projects/99b04f41cebb4755aa0783764c1299aa` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:25:48.702268 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6f9a37-b235-4eda-b0d3-30ca2d54c16f admin admin] RBAC: Authorizing `identity:delete_project(project_id=99b04f41cebb4755aa0783764c1299aa)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:25:48.704467 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6f9a37-b235-4eda-b0d3-30ca2d54c16f admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:25:48.809443 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-0c6f9a37-b235-4eda-b0d3-30ca2d54c16f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '99b04f41cebb4755aa0783764c1299aa', 'request_id': 'req-0c6f9a37-b235-4eda-b0d3-30ca2d54c16f'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:25:48.815266 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-0c6f9a37-b235-4eda-b0d3-30ca2d54c16f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '99b04f41cebb4755aa0783764c1299aa', 'request_id': 'req-0c6f9a37-b235-4eda-b0d3-30ca2d54c16f'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:25:48.907813 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-0c6f9a37-b235-4eda-b0d3-30ca2d54c16f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '99b04f41cebb4755aa0783764c1299aa', 'request_id': 'req-0c6f9a37-b235-4eda-b0d3-30ca2d54c16f'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:25:48.934469 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-0c6f9a37-b235-4eda-b0d3-30ca2d54c16f admin admin] The token cache is being invalidate because project 99b04f41cebb4755aa0783764c1299aa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81750) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 01:25:48.935286 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-0c6f9a37-b235-4eda-b0d3-30ca2d54c16f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:25:48.936700 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-0c6f9a37-b235-4eda-b0d3-30ca2d54c16f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:25:48.942259 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1122/2243] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 01:25:48 2022] DELETE /identity/v3/projects/99b04f41cebb4755aa0783764c1299aa => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:25:55.953134 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484309d3-df4c-4077-bb09-f3a251576c57 None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:25:55.953910 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484309d3-df4c-4077-bb09-f3a251576c57 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:25:55.954062 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484309d3-df4c-4077-bb09-f3a251576c57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:25:55.975154 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-484309d3-df4c-4077-bb09-f3a251576c57 None None] MFA Rules not processed for user `b3efb4f9daac4a0ca05a36dd361f17e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:25:55.976384 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-484309d3-df4c-4077-bb09-f3a251576c57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:25:55.993397 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:25:56.030015 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1122/2244] 10.222.0.20 () {58 vars in 967 bytes} [Wed Aug 31 01:25:55 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:25:56.042384 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6f9a37-b235-4eda-b0d3-30ca2d54c16f admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:25:56.043821 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-0c6f9a37-b235-4eda-b0d3-30ca2d54c16f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:25:56.058800 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:25:56.113100 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6f9a37-b235-4eda-b0d3-30ca2d54c16f admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:25:56.118960 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce6cff5-e6e0-4bcb-b8c4-4354577c7d0c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc1d010919ec40878ae829cc8dceec4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:25:56.119557 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce6cff5-e6e0-4bcb-b8c4-4354577c7d0c service swift] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:25:56.119835 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce6cff5-e6e0-4bcb-b8c4-4354577c7d0c service swift] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:25:56.120108 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce6cff5-e6e0-4bcb-b8c4-4354577c7d0c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:25:56.127842 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce6cff5-e6e0-4bcb-b8c4-4354577c7d0c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:25:56.129462 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce6cff5-e6e0-4bcb-b8c4-4354577c7d0c service swift] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:25:56.137788 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1123/2245] 10.222.0.20 () {60 vars in 1384 bytes} [Wed Aug 31 01:25:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 01:26:04.775293 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63b7a1f-58e5-4fb1-9119-c6f9f614a499 None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:26:04.775829 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63b7a1f-58e5-4fb1-9119-c6f9f614a499 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:26:04.775829 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63b7a1f-58e5-4fb1-9119-c6f9f614a499 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:26:04.795774 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-a63b7a1f-58e5-4fb1-9119-c6f9f614a499 None None] MFA Rules not processed for user `52ed99737fd240789bb682fd9bf11be3`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:26:04.797028 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-a63b7a1f-58e5-4fb1-9119-c6f9f614a499 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:26:04.852437 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1123/2246] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:26:04 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:26:06.175176 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=26 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:26:14.845332 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=40 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:26:15.622495 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97df4c0d-6c54-470b-9044-f2819045c087 None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:26:15.623038 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97df4c0d-6c54-470b-9044-f2819045c087 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:26:15.623698 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97df4c0d-6c54-470b-9044-f2819045c087 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:26:15.642805 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-97df4c0d-6c54-470b-9044-f2819045c087 None None] MFA Rules not processed for user `71d0408df86f4be0ac59cc80575a14e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:26:15.644021 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-97df4c0d-6c54-470b-9044-f2819045c087 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:26:15.699546 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1124/2247] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:26:15 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:26:25.692891 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=25 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:26:49.458037 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61157f26-cdd4-4740-b177-81ac3d3220cf None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:26:49.458963 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61157f26-cdd4-4740-b177-81ac3d3220cf None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:26:49.458963 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61157f26-cdd4-4740-b177-81ac3d3220cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:26:49.479294 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-61157f26-cdd4-4740-b177-81ac3d3220cf None None] MFA Rules not processed for user `efaf57b6482149ca86a9bba26d7654f2`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:26:49.480506 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-61157f26-cdd4-4740-b177-81ac3d3220cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:26:49.537452 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1124/2248] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:26:49 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 01:26:59.530357 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=25 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:27:22.309408 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97df4c0d-6c54-470b-9044-f2819045c087 None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:27:22.310820 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-97df4c0d-6c54-470b-9044-f2819045c087 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:27:22.366881 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97df4c0d-6c54-470b-9044-f2819045c087 None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:27:22.370507 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd1b130-8f3a-4b57-84ac-115443c71988 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0817c3320d5d41d8bba0550c8ae8dd60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:27:22.371107 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd1b130-8f3a-4b57-84ac-115443c71988 service neutron] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:27:22.371387 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd1b130-8f3a-4b57-84ac-115443c71988 service neutron] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:27:22.371659 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd1b130-8f3a-4b57-84ac-115443c71988 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:27:22.373384 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-9bd1b130-8f3a-4b57-84ac-115443c71988 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:27:22.441129 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd1b130-8f3a-4b57-84ac-115443c71988 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:27:22.442599 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd1b130-8f3a-4b57-84ac-115443c71988 service neutron] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:27:22.453270 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1125/2249] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:27:22 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:27:23.128749 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61157f26-cdd4-4740-b177-81ac3d3220cf None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:27:23.138922 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61157f26-cdd4-4740-b177-81ac3d3220cf None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:27:23.142585 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8ca98a-ffc3-4d07-8775-317c21491f32 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0817c3320d5d41d8bba0550c8ae8dd60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:27:23.143173 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8ca98a-ffc3-4d07-8775-317c21491f32 service neutron] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:27:23.143452 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8ca98a-ffc3-4d07-8775-317c21491f32 service neutron] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:27:23.143739 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8ca98a-ffc3-4d07-8775-317c21491f32 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:27:23.145441 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-ad8ca98a-ffc3-4d07-8775-317c21491f32 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:27:23.158462 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8ca98a-ffc3-4d07-8775-317c21491f32 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:27:23.159884 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8ca98a-ffc3-4d07-8775-317c21491f32 service neutron] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:27:23.170840 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1125/2250] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:27:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:27:25.991640 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd1b130-8f3a-4b57-84ac-115443c71988 service neutron] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:27:26.000400 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd1b130-8f3a-4b57-84ac-115443c71988 service neutron] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:27:26.004169 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c8d695-52b5-452b-902f-e61fd4f88cdb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0817c3320d5d41d8bba0550c8ae8dd60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:27:26.004817 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c8d695-52b5-452b-902f-e61fd4f88cdb service neutron] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:27:26.005623 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c8d695-52b5-452b-902f-e61fd4f88cdb service neutron] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:27:26.005895 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c8d695-52b5-452b-902f-e61fd4f88cdb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:27:26.007661 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-a9c8d695-52b5-452b-902f-e61fd4f88cdb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:27:26.077888 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c8d695-52b5-452b-902f-e61fd4f88cdb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:27:26.079605 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c8d695-52b5-452b-902f-e61fd4f88cdb service neutron] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:27:26.090530 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1126/2251] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:27:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:27:30.191425 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fb3082-ff17-4d62-88c4-47b48930d591 None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:27:30.191948 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fb3082-ff17-4d62-88c4-47b48930d591 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:27:30.192150 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fb3082-ff17-4d62-88c4-47b48930d591 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:27:30.224054 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-64fb3082-ff17-4d62-88c4-47b48930d591 None None] MFA Rules not processed for user `52ed99737fd240789bb682fd9bf11be3`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:27:30.225827 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-64fb3082-ff17-4d62-88c4-47b48930d591 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:27:30.232743 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1126/2252] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:27:30 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:27:31.024868 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c8d695-52b5-452b-902f-e61fd4f88cdb service neutron] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:27:31.026452 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-a9c8d695-52b5-452b-902f-e61fd4f88cdb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:27:31.038774 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c8d695-52b5-452b-902f-e61fd4f88cdb service neutron] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:27:31.043725 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f75a808-3e77-4d00-908e-dcc80f97d312 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:27:31.044350 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f75a808-3e77-4d00-908e-dcc80f97d312 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:27:31.044629 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f75a808-3e77-4d00-908e-dcc80f97d312 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:27:31.044929 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f75a808-3e77-4d00-908e-dcc80f97d312 admin admin] PATH_INFO: `/v3/users/71d0408df86f4be0ac59cc80575a14e2` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:27:31.046036 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f75a808-3e77-4d00-908e-dcc80f97d312 admin admin] RBAC: Authorizing `identity:delete_user(user_id=71d0408df86f4be0ac59cc80575a14e2)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:27:31.047940 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f75a808-3e77-4d00-908e-dcc80f97d312 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:27:31.269763 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:27:31.283620 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-6f75a808-3e77-4d00-908e-dcc80f97d312 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '71d0408df86f4be0ac59cc80575a14e2'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:27:31.296377 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-6f75a808-3e77-4d00-908e-dcc80f97d312 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '71d0408df86f4be0ac59cc80575a14e2'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:27:31.297120 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-6f75a808-3e77-4d00-908e-dcc80f97d312 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '71d0408df86f4be0ac59cc80575a14e2'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:27:31.304340 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-6f75a808-3e77-4d00-908e-dcc80f97d312 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '71d0408df86f4be0ac59cc80575a14e2'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:27:31.384940 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-6f75a808-3e77-4d00-908e-dcc80f97d312 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '71d0408df86f4be0ac59cc80575a14e2'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:27:31.391767 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1127/2253] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 01:27:31 2022] DELETE /identity/v3/users/71d0408df86f4be0ac59cc80575a14e2 => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:27:31.599471 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fb3082-ff17-4d62-88c4-47b48930d591 None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:27:31.601227 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-64fb3082-ff17-4d62-88c4-47b48930d591 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:27:31.642899 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f75a808-3e77-4d00-908e-dcc80f97d312 admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:27:31.644411 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-6f75a808-3e77-4d00-908e-dcc80f97d312 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:27:31.719557 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fb3082-ff17-4d62-88c4-47b48930d591 None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:27:31.743406 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f75a808-3e77-4d00-908e-dcc80f97d312 admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:27:31.748107 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017ea5a3-3763-4e57-ad07-ca31d748436d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:27:31.748762 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017ea5a3-3763-4e57-ad07-ca31d748436d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:27:31.749046 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017ea5a3-3763-4e57-ad07-ca31d748436d admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:27:31.749319 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017ea5a3-3763-4e57-ad07-ca31d748436d admin admin] PATH_INFO: `/v3/projects/d58414b1dd2342d58cc7665aee50b86b` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:27:31.750570 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017ea5a3-3763-4e57-ad07-ca31d748436d admin admin] RBAC: Authorizing `identity:delete_project(project_id=d58414b1dd2342d58cc7665aee50b86b)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:27:31.752487 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017ea5a3-3763-4e57-ad07-ca31d748436d admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:27:31.790231 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71283fc-9b5d-4dd1-a743-240af022a9ec service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dea0fe5b1d6b405cb47b95192580303b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:27:31.790583 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71283fc-9b5d-4dd1-a743-240af022a9ec service placement] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:27:31.791042 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71283fc-9b5d-4dd1-a743-240af022a9ec service placement] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:27:31.791323 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71283fc-9b5d-4dd1-a743-240af022a9ec service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:27:31.794208 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-d71283fc-9b5d-4dd1-a743-240af022a9ec service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:27:31.807531 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71283fc-9b5d-4dd1-a743-240af022a9ec service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:27:31.809190 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71283fc-9b5d-4dd1-a743-240af022a9ec service placement] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:27:31.821009 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1127/2254] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:27:31 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:27:31.907815 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-017ea5a3-3763-4e57-ad07-ca31d748436d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd58414b1dd2342d58cc7665aee50b86b', 'request_id': 'req-017ea5a3-3763-4e57-ad07-ca31d748436d'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:27:31.918055 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-017ea5a3-3763-4e57-ad07-ca31d748436d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd58414b1dd2342d58cc7665aee50b86b', 'request_id': 'req-017ea5a3-3763-4e57-ad07-ca31d748436d'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:27:32.007929 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-017ea5a3-3763-4e57-ad07-ca31d748436d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd58414b1dd2342d58cc7665aee50b86b', 'request_id': 'req-017ea5a3-3763-4e57-ad07-ca31d748436d'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:27:32.024029 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-017ea5a3-3763-4e57-ad07-ca31d748436d admin admin] The token cache is being invalidate because project d58414b1dd2342d58cc7665aee50b86b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81750) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 01:27:32.024763 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-017ea5a3-3763-4e57-ad07-ca31d748436d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:27:32.025658 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-017ea5a3-3763-4e57-ad07-ca31d748436d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:27:32.029249 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1128/2255] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 01:27:31 2022] DELETE /identity/v3/projects/d58414b1dd2342d58cc7665aee50b86b => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 01:27:32.033987 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b71a6e-6a31-4723-84c7-a657a0685b2c None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:27:32.034400 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b71a6e-6a31-4723-84c7-a657a0685b2c None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:27:32.034785 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b71a6e-6a31-4723-84c7-a657a0685b2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:27:32.059860 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-15b71a6e-6a31-4723-84c7-a657a0685b2c None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:27:32.061839 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-15b71a6e-6a31-4723-84c7-a657a0685b2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:27:32.137867 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1128/2256] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:27:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:27:32.141862 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7a32b7-9e9c-4110-8a7d-0cbf8e0a570a None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:27:32.142845 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7a32b7-9e9c-4110-8a7d-0cbf8e0a570a None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:27:32.142845 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7a32b7-9e9c-4110-8a7d-0cbf8e0a570a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:27:32.164367 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-8b7a32b7-9e9c-4110-8a7d-0cbf8e0a570a None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:27:32.165851 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-8b7a32b7-9e9c-4110-8a7d-0cbf8e0a570a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:27:32.175416 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1129/2257] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:27:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:27:32.180690 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b71a6e-6a31-4723-84c7-a657a0685b2c None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:27:32.190466 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b71a6e-6a31-4723-84c7-a657a0685b2c None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:27:32.195727 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3838d580-fe27-4f3d-a350-ff33ac9235d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:27:32.197450 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3838d580-fe27-4f3d-a350-ff33ac9235d5 admin admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:27:32.197450 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3838d580-fe27-4f3d-a350-ff33ac9235d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:27:32.197450 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3838d580-fe27-4f3d-a350-ff33ac9235d5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:27:32.197715 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3838d580-fe27-4f3d-a350-ff33ac9235d5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:27:32.197852 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3838d580-fe27-4f3d-a350-ff33ac9235d5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:27:32.199420 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3838d580-fe27-4f3d-a350-ff33ac9235d5 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:27:32.208655 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:27:32.215080 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1129/2258] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Aug 31 01:27:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:27:32.220737 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7a32b7-9e9c-4110-8a7d-0cbf8e0a570a None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:27:32.229781 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7a32b7-9e9c-4110-8a7d-0cbf8e0a570a None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:27:32.235868 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8286434-f195-4770-b956-5a0a76706e19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:27:32.237013 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8286434-f195-4770-b956-5a0a76706e19 admin admin] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:27:32.237558 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8286434-f195-4770-b956-5a0a76706e19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:27:32.238204 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8286434-f195-4770-b956-5a0a76706e19 admin admin] PATH_INFO: `/v3/projects` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:27:32.240021 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8286434-f195-4770-b956-5a0a76706e19 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:27:32.244215 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8286434-f195-4770-b956-5a0a76706e19 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:27:32.350369 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1130/2259] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 01:27:32 2022] POST /identity/v3/projects => generated 373 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 01:27:32.354286 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3838d580-fe27-4f3d-a350-ff33ac9235d5 admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:27:32.367371 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3838d580-fe27-4f3d-a350-ff33ac9235d5 admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:27:32.433845 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb505e1-81aa-46b9-bc53-e09ad92033bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:27:32.434511 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb505e1-81aa-46b9-bc53-e09ad92033bb admin admin] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:27:32.434792 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb505e1-81aa-46b9-bc53-e09ad92033bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:27:32.435073 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb505e1-81aa-46b9-bc53-e09ad92033bb admin admin] PATH_INFO: `/v3/users` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:27:32.435710 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb505e1-81aa-46b9-bc53-e09ad92033bb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:27:32.437827 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb505e1-81aa-46b9-bc53-e09ad92033bb admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:27:32.513081 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1130/2260] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 01:27:32 2022] POST /identity/v3/users => generated 287 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:27:32.518263 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8286434-f195-4770-b956-5a0a76706e19 admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:27:32.530617 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8286434-f195-4770-b956-5a0a76706e19 admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:27:32.535406 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e29812-b5a2-495f-aee0-c4afe709c9ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:27:32.536026 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e29812-b5a2-495f-aee0-c4afe709c9ac admin admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:27:32.536303 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e29812-b5a2-495f-aee0-c4afe709c9ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:27:32.536575 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e29812-b5a2-495f-aee0-c4afe709c9ac admin admin] PATH_INFO: `/v3/roles` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:27:32.537291 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e29812-b5a2-495f-aee0-c4afe709c9ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:27:32.538890 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e29812-b5a2-495f-aee0-c4afe709c9ac admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:27:32.541163 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:27:32.554405 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1131/2261] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:27:32 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:27:32.554585 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb505e1-81aa-46b9-bc53-e09ad92033bb admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:27:32.564690 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb505e1-81aa-46b9-bc53-e09ad92033bb admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:27:32.570553 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2ba498-c92a-464f-be46-e8d24b8b2bfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:27:32.571234 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2ba498-c92a-464f-be46-e8d24b8b2bfc admin admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:27:32.571586 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2ba498-c92a-464f-be46-e8d24b8b2bfc admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:27:32.571973 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2ba498-c92a-464f-be46-e8d24b8b2bfc admin admin] PATH_INFO: `/v3/roles` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:27:32.572683 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2ba498-c92a-464f-be46-e8d24b8b2bfc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:27:32.577449 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2ba498-c92a-464f-be46-e8d24b8b2bfc admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:27:32.586620 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1131/2262] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:27:32 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:27:32.590624 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e29812-b5a2-495f-aee0-c4afe709c9ac admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:27:32.603163 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e29812-b5a2-495f-aee0-c4afe709c9ac admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:27:32.606760 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fbe72e-7cec-40dd-86a7-9a12d749a931 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:27:32.607544 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fbe72e-7cec-40dd-86a7-9a12d749a931 admin admin] REQUEST_METHOD: `PUT` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:27:32.608516 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fbe72e-7cec-40dd-86a7-9a12d749a931 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:27:32.608906 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fbe72e-7cec-40dd-86a7-9a12d749a931 admin admin] PATH_INFO: `/v3/projects/4a9fd8516beb4d03b026b9e23f560ed3/users/a6f4a501d522451798a9fc4a45103b41/roles/d0b9768f4d014766b9326387f2e11f34` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:27:32.626580 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fbe72e-7cec-40dd-86a7-9a12d749a931 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a9fd8516beb4d03b026b9e23f560ed3, user_id=a6f4a501d522451798a9fc4a45103b41, role_id=d0b9768f4d014766b9326387f2e11f34)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:27:32.631663 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fbe72e-7cec-40dd-86a7-9a12d749a931 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:27:32.744476 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1132/2263] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 01:27:32 2022] PUT /identity/v3/projects/4a9fd8516beb4d03b026b9e23f560ed3/users/a6f4a501d522451798a9fc4a45103b41/roles/d0b9768f4d014766b9326387f2e11f34 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:27:32.755611 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86712902-df3e-4ed2-853f-bb62079008a9 None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:27:32.756155 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86712902-df3e-4ed2-853f-bb62079008a9 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:27:32.756155 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86712902-df3e-4ed2-853f-bb62079008a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:27:32.798218 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-86712902-df3e-4ed2-853f-bb62079008a9 None None] MFA Rules not processed for user `a6f4a501d522451798a9fc4a45103b41`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:27:32.799683 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-86712902-df3e-4ed2-853f-bb62079008a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:27:32.875904 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1132/2264] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:27:32 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:27:32.881111 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fbe72e-7cec-40dd-86a7-9a12d749a931 admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:27:32.950345 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fbe72e-7cec-40dd-86a7-9a12d749a931 admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:27:32.955991 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51def5fd-cc31-46b4-9c4e-7806af2b0531 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:27:32.956614 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51def5fd-cc31-46b4-9c4e-7806af2b0531 admin admin] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:27:32.956929 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51def5fd-cc31-46b4-9c4e-7806af2b0531 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:27:32.957210 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51def5fd-cc31-46b4-9c4e-7806af2b0531 admin admin] PATH_INFO: `/v3/projects` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:27:32.957886 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51def5fd-cc31-46b4-9c4e-7806af2b0531 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:27:32.959956 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51def5fd-cc31-46b4-9c4e-7806af2b0531 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:27:33.063718 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1133/2265] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 01:27:32 2022] POST /identity/v3/projects => generated 373 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:27:33.069127 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86712902-df3e-4ed2-853f-bb62079008a9 None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:27:33.085145 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86712902-df3e-4ed2-853f-bb62079008a9 None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:27:33.142768 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5ed468-cacb-4085-89ae-1c0e73100be2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:27:33.143389 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5ed468-cacb-4085-89ae-1c0e73100be2 admin admin] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:27:33.143680 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5ed468-cacb-4085-89ae-1c0e73100be2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:27:33.143964 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5ed468-cacb-4085-89ae-1c0e73100be2 admin admin] PATH_INFO: `/v3/users` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:27:33.144599 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5ed468-cacb-4085-89ae-1c0e73100be2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:27:33.146892 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5ed468-cacb-4085-89ae-1c0e73100be2 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:27:33.208144 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:27:33.214322 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1133/2266] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 01:27:33 2022] POST /identity/v3/users => generated 293 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:27:33.218638 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51def5fd-cc31-46b4-9c4e-7806af2b0531 admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:27:33.230506 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51def5fd-cc31-46b4-9c4e-7806af2b0531 admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:27:33.235145 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79de4119-70c9-402e-83a2-c09b78c4ff0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:27:33.237698 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79de4119-70c9-402e-83a2-c09b78c4ff0f admin admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:27:33.237698 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79de4119-70c9-402e-83a2-c09b78c4ff0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:27:33.237698 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79de4119-70c9-402e-83a2-c09b78c4ff0f admin admin] PATH_INFO: `/v3/roles` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:27:33.237698 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79de4119-70c9-402e-83a2-c09b78c4ff0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:27:33.238532 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79de4119-70c9-402e-83a2-c09b78c4ff0f admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:27:33.249471 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1134/2267] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:27:33 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:27:33.255052 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5ed468-cacb-4085-89ae-1c0e73100be2 admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:27:33.263841 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5ed468-cacb-4085-89ae-1c0e73100be2 admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:27:33.269360 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a94869-590f-438f-bcc4-dc05a44b59d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:27:33.270049 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a94869-590f-438f-bcc4-dc05a44b59d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:27:33.274231 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a94869-590f-438f-bcc4-dc05a44b59d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:27:33.274231 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a94869-590f-438f-bcc4-dc05a44b59d3 admin admin] PATH_INFO: `/v3/projects/beb2b0d6d69f405d96b9f0f132bbbfa8/users/f13b28114f224dbb99114c7d09369c19/roles/d009411496b8452b88e77ffb8db015a7` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:27:33.295221 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a94869-590f-438f-bcc4-dc05a44b59d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=beb2b0d6d69f405d96b9f0f132bbbfa8, user_id=f13b28114f224dbb99114c7d09369c19, role_id=d009411496b8452b88e77ffb8db015a7)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:27:33.298113 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a94869-590f-438f-bcc4-dc05a44b59d3 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:27:33.375774 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1134/2268] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 01:27:33 2022] PUT /identity/v3/projects/beb2b0d6d69f405d96b9f0f132bbbfa8/users/f13b28114f224dbb99114c7d09369c19/roles/d009411496b8452b88e77ffb8db015a7 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:27:33.378000 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79de4119-70c9-402e-83a2-c09b78c4ff0f admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:27:33.388266 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79de4119-70c9-402e-83a2-c09b78c4ff0f admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:27:33.459554 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef22f5cc-f45a-4f56-9f85-e5eb4f1c1c9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:27:33.460297 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef22f5cc-f45a-4f56-9f85-e5eb4f1c1c9c admin admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:27:33.460596 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef22f5cc-f45a-4f56-9f85-e5eb4f1c1c9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:27:33.460899 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef22f5cc-f45a-4f56-9f85-e5eb4f1c1c9c admin admin] PATH_INFO: `/v3/roles` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:27:33.461540 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef22f5cc-f45a-4f56-9f85-e5eb4f1c1c9c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:27:33.463148 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef22f5cc-f45a-4f56-9f85-e5eb4f1c1c9c admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:27:33.474013 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1135/2269] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:27:33 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:27:33.478460 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a94869-590f-438f-bcc4-dc05a44b59d3 admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:27:33.490027 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a94869-590f-438f-bcc4-dc05a44b59d3 admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:27:33.494986 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34488ea-5309-4561-8b62-9f8836ba622f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:27:33.495654 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34488ea-5309-4561-8b62-9f8836ba622f admin admin] REQUEST_METHOD: `PUT` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:27:33.495964 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34488ea-5309-4561-8b62-9f8836ba622f admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:27:33.496247 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34488ea-5309-4561-8b62-9f8836ba622f admin admin] PATH_INFO: `/v3/projects/beb2b0d6d69f405d96b9f0f132bbbfa8/users/f13b28114f224dbb99114c7d09369c19/roles/d009411496b8452b88e77ffb8db015a7` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:27:33.498332 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34488ea-5309-4561-8b62-9f8836ba622f admin admin] RBAC: Authorizing `identity:create_grant(project_id=beb2b0d6d69f405d96b9f0f132bbbfa8, user_id=f13b28114f224dbb99114c7d09369c19, role_id=d009411496b8452b88e77ffb8db015a7)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:27:33.501480 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34488ea-5309-4561-8b62-9f8836ba622f admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:27:33.512446 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1135/2270] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 01:27:33 2022] PUT /identity/v3/projects/beb2b0d6d69f405d96b9f0f132bbbfa8/users/f13b28114f224dbb99114c7d09369c19/roles/d009411496b8452b88e77ffb8db015a7 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:27:33.524920 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2807cb45-e7c3-4e21-a5c3-0506afd28f4c None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:27:33.525223 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2807cb45-e7c3-4e21-a5c3-0506afd28f4c None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:27:33.525507 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2807cb45-e7c3-4e21-a5c3-0506afd28f4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:27:33.555366 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-2807cb45-e7c3-4e21-a5c3-0506afd28f4c None None] MFA Rules not processed for user `f13b28114f224dbb99114c7d09369c19`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:27:33.556675 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-2807cb45-e7c3-4e21-a5c3-0506afd28f4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:27:33.590354 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:27:33.626695 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1136/2271] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:27:33 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:27:33.637987 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34488ea-5309-4561-8b62-9f8836ba622f admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:27:33.639635 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-e34488ea-5309-4561-8b62-9f8836ba622f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:27:33.719023 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34488ea-5309-4561-8b62-9f8836ba622f admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:27:33.723268 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea5e950-d4a8-45d3-bf23-01f58240a05b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5c8fc69c518412ba1aad61a9f4f558c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:27:33.724639 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea5e950-d4a8-45d3-bf23-01f58240a05b service cinder] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:27:33.725069 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea5e950-d4a8-45d3-bf23-01f58240a05b service cinder] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:27:33.725947 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea5e950-d4a8-45d3-bf23-01f58240a05b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:27:33.736191 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea5e950-d4a8-45d3-bf23-01f58240a05b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:27:33.737675 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea5e950-d4a8-45d3-bf23-01f58240a05b service cinder] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:27:33.748194 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1136/2272] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:27:33 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:27:34.842113 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2807cb45-e7c3-4e21-a5c3-0506afd28f4c None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:27:34.931032 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2807cb45-e7c3-4e21-a5c3-0506afd28f4c None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:27:34.936320 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79359e67-42ab-4247-973e-7a826732e023 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:27:34.937862 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79359e67-42ab-4247-973e-7a826732e023 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:27:34.938264 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79359e67-42ab-4247-973e-7a826732e023 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:27:34.938666 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79359e67-42ab-4247-973e-7a826732e023 admin admin] PATH_INFO: `/v3/users/a6f4a501d522451798a9fc4a45103b41` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:27:34.939973 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79359e67-42ab-4247-973e-7a826732e023 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a6f4a501d522451798a9fc4a45103b41)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:27:34.942293 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79359e67-42ab-4247-973e-7a826732e023 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:27:35.156569 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-79359e67-42ab-4247-973e-7a826732e023 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a6f4a501d522451798a9fc4a45103b41'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:27:35.172783 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-79359e67-42ab-4247-973e-7a826732e023 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a6f4a501d522451798a9fc4a45103b41'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:27:35.173746 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-79359e67-42ab-4247-973e-7a826732e023 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a6f4a501d522451798a9fc4a45103b41'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:27:35.197598 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-79359e67-42ab-4247-973e-7a826732e023 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a6f4a501d522451798a9fc4a45103b41'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:27:35.276254 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-79359e67-42ab-4247-973e-7a826732e023 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a6f4a501d522451798a9fc4a45103b41'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:27:35.284196 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1137/2273] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 01:27:34 2022] DELETE /identity/v3/users/a6f4a501d522451798a9fc4a45103b41 => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:27:35.289290 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd62c7e-9a5c-47cc-88ae-70b644026448 None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:27:35.290116 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd62c7e-9a5c-47cc-88ae-70b644026448 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:27:35.290417 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd62c7e-9a5c-47cc-88ae-70b644026448 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:27:35.309714 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-1fd62c7e-9a5c-47cc-88ae-70b644026448 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:27:35.311016 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-1fd62c7e-9a5c-47cc-88ae-70b644026448 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:27:35.374431 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1137/2274] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:27:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:27:35.385067 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79359e67-42ab-4247-973e-7a826732e023 admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:27:35.386512 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-79359e67-42ab-4247-973e-7a826732e023 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:27:35.444269 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:27:35.448478 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79359e67-42ab-4247-973e-7a826732e023 admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:27:35.452072 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cbb661-5a97-4f20-8a1e-26c299947a07 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0817c3320d5d41d8bba0550c8ae8dd60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:27:35.452707 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cbb661-5a97-4f20-8a1e-26c299947a07 service neutron] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:27:35.459681 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cbb661-5a97-4f20-8a1e-26c299947a07 service neutron] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:27:35.459681 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cbb661-5a97-4f20-8a1e-26c299947a07 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:27:35.463258 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cbb661-5a97-4f20-8a1e-26c299947a07 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:27:35.464674 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cbb661-5a97-4f20-8a1e-26c299947a07 service neutron] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:27:35.474462 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1138/2275] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:27:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:27:35.905503 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd62c7e-9a5c-47cc-88ae-70b644026448 None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:27:35.906999 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-1fd62c7e-9a5c-47cc-88ae-70b644026448 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:27:35.926910 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd62c7e-9a5c-47cc-88ae-70b644026448 None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:27:35.935287 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1357a853-a6ee-4e68-8bc8-ab088f9f2726 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:27:35.935949 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1357a853-a6ee-4e68-8bc8-ab088f9f2726 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:27:35.936234 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1357a853-a6ee-4e68-8bc8-ab088f9f2726 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:27:35.936512 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1357a853-a6ee-4e68-8bc8-ab088f9f2726 admin admin] PATH_INFO: `/v3/projects/4a9fd8516beb4d03b026b9e23f560ed3` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:27:35.937991 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1357a853-a6ee-4e68-8bc8-ab088f9f2726 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a9fd8516beb4d03b026b9e23f560ed3)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:27:35.939834 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1357a853-a6ee-4e68-8bc8-ab088f9f2726 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:27:36.059988 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-1357a853-a6ee-4e68-8bc8-ab088f9f2726 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a9fd8516beb4d03b026b9e23f560ed3', 'request_id': 'req-1357a853-a6ee-4e68-8bc8-ab088f9f2726'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:27:36.065664 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-1357a853-a6ee-4e68-8bc8-ab088f9f2726 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a9fd8516beb4d03b026b9e23f560ed3', 'request_id': 'req-1357a853-a6ee-4e68-8bc8-ab088f9f2726'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:27:36.066736 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=92,keystone:INSERT=2,keystone:DELETE=6 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:27:36.072158 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-1357a853-a6ee-4e68-8bc8-ab088f9f2726 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a9fd8516beb4d03b026b9e23f560ed3', 'request_id': 'req-1357a853-a6ee-4e68-8bc8-ab088f9f2726'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:27:36.167848 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-1357a853-a6ee-4e68-8bc8-ab088f9f2726 admin admin] The token cache is being invalidate because project 4a9fd8516beb4d03b026b9e23f560ed3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81749) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 01:27:36.168268 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-1357a853-a6ee-4e68-8bc8-ab088f9f2726 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:27:36.168803 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-1357a853-a6ee-4e68-8bc8-ab088f9f2726 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:27:36.170319 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1138/2276] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 01:27:35 2022] DELETE /identity/v3/projects/4a9fd8516beb4d03b026b9e23f560ed3 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:27:36.174628 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cbb661-5a97-4f20-8a1e-26c299947a07 service neutron] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:27:36.188124 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cbb661-5a97-4f20-8a1e-26c299947a07 service neutron] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:27:36.291129 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f1b1db-afed-4816-aac2-5da41dddb7ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:27:36.291865 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f1b1db-afed-4816-aac2-5da41dddb7ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:27:36.292272 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f1b1db-afed-4816-aac2-5da41dddb7ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:27:36.292643 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f1b1db-afed-4816-aac2-5da41dddb7ce admin admin] PATH_INFO: `/v3/users/f13b28114f224dbb99114c7d09369c19` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:27:36.294763 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f1b1db-afed-4816-aac2-5da41dddb7ce admin admin] RBAC: Authorizing `identity:delete_user(user_id=f13b28114f224dbb99114c7d09369c19)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:27:36.296836 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f1b1db-afed-4816-aac2-5da41dddb7ce admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:27:36.587575 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-36f1b1db-afed-4816-aac2-5da41dddb7ce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f13b28114f224dbb99114c7d09369c19'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:27:36.600544 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-36f1b1db-afed-4816-aac2-5da41dddb7ce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f13b28114f224dbb99114c7d09369c19'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:27:36.601268 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-36f1b1db-afed-4816-aac2-5da41dddb7ce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f13b28114f224dbb99114c7d09369c19'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:27:36.608496 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-36f1b1db-afed-4816-aac2-5da41dddb7ce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f13b28114f224dbb99114c7d09369c19'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:27:36.685004 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-36f1b1db-afed-4816-aac2-5da41dddb7ce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f13b28114f224dbb99114c7d09369c19'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:27:36.691451 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1139/2277] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 01:27:36 2022] DELETE /identity/v3/users/f13b28114f224dbb99114c7d09369c19 => generated 0 bytes in 517 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:27:37.174261 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1357a853-a6ee-4e68-8bc8-ab088f9f2726 admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:27:37.175811 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-1357a853-a6ee-4e68-8bc8-ab088f9f2726 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:27:37.190543 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1357a853-a6ee-4e68-8bc8-ab088f9f2726 admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:27:37.251163 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d60dd5f-b573-4764-ac09-68618c6dca93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:27:37.251800 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d60dd5f-b573-4764-ac09-68618c6dca93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:27:37.252078 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d60dd5f-b573-4764-ac09-68618c6dca93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:27:37.252544 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d60dd5f-b573-4764-ac09-68618c6dca93 admin admin] PATH_INFO: `/v3/projects/beb2b0d6d69f405d96b9f0f132bbbfa8` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:27:37.253750 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d60dd5f-b573-4764-ac09-68618c6dca93 admin admin] RBAC: Authorizing `identity:delete_project(project_id=beb2b0d6d69f405d96b9f0f132bbbfa8)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:27:37.256376 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d60dd5f-b573-4764-ac09-68618c6dca93 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:27:37.361706 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-2d60dd5f-b573-4764-ac09-68618c6dca93 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'beb2b0d6d69f405d96b9f0f132bbbfa8', 'request_id': 'req-2d60dd5f-b573-4764-ac09-68618c6dca93'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:27:37.367768 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-2d60dd5f-b573-4764-ac09-68618c6dca93 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'beb2b0d6d69f405d96b9f0f132bbbfa8', 'request_id': 'req-2d60dd5f-b573-4764-ac09-68618c6dca93'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:27:37.373896 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-2d60dd5f-b573-4764-ac09-68618c6dca93 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'beb2b0d6d69f405d96b9f0f132bbbfa8', 'request_id': 'req-2d60dd5f-b573-4764-ac09-68618c6dca93'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:27:37.411357 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218e3950-a11f-426b-b15b-85d1d2e5b9cb None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:27:37.411860 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218e3950-a11f-426b-b15b-85d1d2e5b9cb None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:27:37.411957 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218e3950-a11f-426b-b15b-85d1d2e5b9cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:27:37.430726 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-218e3950-a11f-426b-b15b-85d1d2e5b9cb None None] MFA Rules not processed for user `1cc3db55df284cd39f28fcfefc619fda`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:27:37.431923 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-218e3950-a11f-426b-b15b-85d1d2e5b9cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:27:37.484535 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-2d60dd5f-b573-4764-ac09-68618c6dca93 admin admin] The token cache is being invalidate because project beb2b0d6d69f405d96b9f0f132bbbfa8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81749) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 01:27:37.485070 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-2d60dd5f-b573-4764-ac09-68618c6dca93 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:27:37.486025 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-2d60dd5f-b573-4764-ac09-68618c6dca93 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:27:37.488436 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1139/2278] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 01:27:37 2022] DELETE /identity/v3/projects/beb2b0d6d69f405d96b9f0f132bbbfa8 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:27:37.492013 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1140/2279] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:27:37 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 01:27:37.497369 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0605d1a2-587c-4375-81fc-6ec71b1512f9 None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:27:37.497658 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0605d1a2-587c-4375-81fc-6ec71b1512f9 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:27:37.497925 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0605d1a2-587c-4375-81fc-6ec71b1512f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:27:37.518637 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-0605d1a2-587c-4375-81fc-6ec71b1512f9 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:27:37.519922 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-0605d1a2-587c-4375-81fc-6ec71b1512f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:27:37.591609 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1140/2280] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:27:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:27:37.598665 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef52b2ee-806e-4b4a-ad8b-e9149b00c1ec None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:27:37.598952 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef52b2ee-806e-4b4a-ad8b-e9149b00c1ec None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:27:37.599231 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef52b2ee-806e-4b4a-ad8b-e9149b00c1ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:27:37.619334 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-ef52b2ee-806e-4b4a-ad8b-e9149b00c1ec None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:27:37.620745 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-ef52b2ee-806e-4b4a-ad8b-e9149b00c1ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:27:37.628978 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1141/2281] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:27:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:27:37.633648 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0605d1a2-587c-4375-81fc-6ec71b1512f9 None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:27:37.643360 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0605d1a2-587c-4375-81fc-6ec71b1512f9 None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:27:37.648673 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f3caa0-f174-4f35-b348-566d28a949f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:27:37.649386 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f3caa0-f174-4f35-b348-566d28a949f4 admin admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:27:37.649738 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f3caa0-f174-4f35-b348-566d28a949f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:27:37.650085 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f3caa0-f174-4f35-b348-566d28a949f4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:27:37.650807 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f3caa0-f174-4f35-b348-566d28a949f4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:27:37.651184 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f3caa0-f174-4f35-b348-566d28a949f4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:27:37.652900 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f3caa0-f174-4f35-b348-566d28a949f4 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:27:37.665435 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1141/2282] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Aug 31 01:27:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:27:37.670048 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef52b2ee-806e-4b4a-ad8b-e9149b00c1ec None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:27:37.679546 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef52b2ee-806e-4b4a-ad8b-e9149b00c1ec None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:27:37.684447 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1ae650-b4f1-4795-be6b-b9e8adfd9bfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:27:37.685177 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1ae650-b4f1-4795-be6b-b9e8adfd9bfd admin admin] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:27:37.685528 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1ae650-b4f1-4795-be6b-b9e8adfd9bfd admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:27:37.686042 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1ae650-b4f1-4795-be6b-b9e8adfd9bfd admin admin] PATH_INFO: `/v3/projects` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:27:37.686591 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1ae650-b4f1-4795-be6b-b9e8adfd9bfd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:27:37.688614 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1ae650-b4f1-4795-be6b-b9e8adfd9bfd admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:27:37.694693 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:27:37.815077 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1142/2283] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 01:27:37 2022] POST /identity/v3/projects => generated 365 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:27:37.819418 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f3caa0-f174-4f35-b348-566d28a949f4 admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:27:37.825947 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:27:37.830512 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f3caa0-f174-4f35-b348-566d28a949f4 admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:27:37.881866 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1ae650-b4f1-4795-be6b-b9e8adfd9bfd admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:27:37.883574 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-ce1ae650-b4f1-4795-be6b-b9e8adfd9bfd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:27:37.887192 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dc06d3-1290-474a-9981-4bd54981a1cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:27:37.887810 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dc06d3-1290-474a-9981-4bd54981a1cd admin admin] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:27:37.888099 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dc06d3-1290-474a-9981-4bd54981a1cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:27:37.888375 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dc06d3-1290-474a-9981-4bd54981a1cd admin admin] PATH_INFO: `/v3/users` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:27:37.889266 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30dc06d3-1290-474a-9981-4bd54981a1cd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:27:37.895862 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30dc06d3-1290-474a-9981-4bd54981a1cd admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:27:37.943601 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1ae650-b4f1-4795-be6b-b9e8adfd9bfd admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:27:37.947291 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4d4155-f045-4800-b729-cb0551a3f756 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0817c3320d5d41d8bba0550c8ae8dd60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:27:37.947892 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4d4155-f045-4800-b729-cb0551a3f756 service neutron] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:27:37.948172 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4d4155-f045-4800-b729-cb0551a3f756 service neutron] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:27:37.948446 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4d4155-f045-4800-b729-cb0551a3f756 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:27:37.950212 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-2c4d4155-f045-4800-b729-cb0551a3f756 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:27:37.960068 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4d4155-f045-4800-b729-cb0551a3f756 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:27:37.961511 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4d4155-f045-4800-b729-cb0551a3f756 service neutron] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:27:37.969375 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1143/2284] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:27:37 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 01:27:38.001987 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1142/2285] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 01:27:37 2022] POST /identity/v3/users => generated 283 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:27:38.006128 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4d4155-f045-4800-b729-cb0551a3f756 service neutron] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:27:38.017735 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4d4155-f045-4800-b729-cb0551a3f756 service neutron] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:27:38.023146 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612fbd28-435c-4151-b2e4-e2a809938a7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:27:38.023745 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612fbd28-435c-4151-b2e4-e2a809938a7a admin admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:27:38.024025 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612fbd28-435c-4151-b2e4-e2a809938a7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:27:38.024297 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612fbd28-435c-4151-b2e4-e2a809938a7a admin admin] PATH_INFO: `/v3/roles` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:27:38.025022 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612fbd28-435c-4151-b2e4-e2a809938a7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:27:38.026624 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612fbd28-435c-4151-b2e4-e2a809938a7a admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:27:38.037947 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1144/2286] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:27:38 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:27:38.042071 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dc06d3-1290-474a-9981-4bd54981a1cd admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:27:38.053120 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dc06d3-1290-474a-9981-4bd54981a1cd admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:27:38.058269 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21013dbb-9463-460f-9123-00a63cd14865 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:27:38.058941 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21013dbb-9463-460f-9123-00a63cd14865 admin admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:27:38.059296 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21013dbb-9463-460f-9123-00a63cd14865 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:27:38.059742 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21013dbb-9463-460f-9123-00a63cd14865 admin admin] PATH_INFO: `/v3/roles` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:27:38.060458 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21013dbb-9463-460f-9123-00a63cd14865 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:27:38.062189 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21013dbb-9463-460f-9123-00a63cd14865 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:27:38.073561 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1143/2287] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:27:38 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:27:38.077707 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612fbd28-435c-4151-b2e4-e2a809938a7a admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:27:38.088669 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612fbd28-435c-4151-b2e4-e2a809938a7a admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:27:38.093909 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc2c861-0089-4737-93cf-1b24c4e36dfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:27:38.094575 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc2c861-0089-4737-93cf-1b24c4e36dfa admin admin] REQUEST_METHOD: `PUT` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:27:38.094857 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc2c861-0089-4737-93cf-1b24c4e36dfa admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:27:38.095133 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc2c861-0089-4737-93cf-1b24c4e36dfa admin admin] PATH_INFO: `/v3/projects/9ac94cad39a5413eabcc9b00306f9f16/users/d7a3e2d79c1d4d44bd88471e4c18d58f/roles/d0b9768f4d014766b9326387f2e11f34` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:27:38.111056 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc2c861-0089-4737-93cf-1b24c4e36dfa admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ac94cad39a5413eabcc9b00306f9f16, user_id=d7a3e2d79c1d4d44bd88471e4c18d58f, role_id=d0b9768f4d014766b9326387f2e11f34)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:27:38.113797 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc2c861-0089-4737-93cf-1b24c4e36dfa admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:27:38.185291 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1145/2288] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 01:27:38 2022] PUT /identity/v3/projects/9ac94cad39a5413eabcc9b00306f9f16/users/d7a3e2d79c1d4d44bd88471e4c18d58f/roles/d0b9768f4d014766b9326387f2e11f34 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:27:38.196861 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4318d474-ff8c-417a-b05f-e892a3510821 None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:27:38.197387 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4318d474-ff8c-417a-b05f-e892a3510821 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:27:38.197387 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4318d474-ff8c-417a-b05f-e892a3510821 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:27:38.228171 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-4318d474-ff8c-417a-b05f-e892a3510821 None None] MFA Rules not processed for user `d7a3e2d79c1d4d44bd88471e4c18d58f`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:27:38.229487 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-4318d474-ff8c-417a-b05f-e892a3510821 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:27:38.301380 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1144/2289] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:27:38 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:27:38.305900 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc2c861-0089-4737-93cf-1b24c4e36dfa admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:27:38.373295 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc2c861-0089-4737-93cf-1b24c4e36dfa admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:27:38.378123 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22bcf4f-165b-408c-a1b8-d3f39086f4f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:27:38.378767 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22bcf4f-165b-408c-a1b8-d3f39086f4f3 admin admin] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:27:38.379053 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22bcf4f-165b-408c-a1b8-d3f39086f4f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:27:38.379817 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22bcf4f-165b-408c-a1b8-d3f39086f4f3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:27:38.380452 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22bcf4f-165b-408c-a1b8-d3f39086f4f3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:27:38.382439 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22bcf4f-165b-408c-a1b8-d3f39086f4f3 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:27:38.545994 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1146/2290] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 01:27:38 2022] POST /identity/v3/projects => generated 363 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:27:38.549790 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4318d474-ff8c-417a-b05f-e892a3510821 None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:27:38.562633 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4318d474-ff8c-417a-b05f-e892a3510821 None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:27:38.634358 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0885ef-34ee-44d5-a8ba-52d6a7eb852f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:27:38.635212 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0885ef-34ee-44d5-a8ba-52d6a7eb852f admin admin] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:27:38.635568 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0885ef-34ee-44d5-a8ba-52d6a7eb852f admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:27:38.635916 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0885ef-34ee-44d5-a8ba-52d6a7eb852f admin admin] PATH_INFO: `/v3/users` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:27:38.636641 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0885ef-34ee-44d5-a8ba-52d6a7eb852f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:27:38.638632 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0885ef-34ee-44d5-a8ba-52d6a7eb852f admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:27:38.741299 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1145/2291] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 01:27:38 2022] POST /identity/v3/users => generated 288 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:27:38.745279 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22bcf4f-165b-408c-a1b8-d3f39086f4f3 admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:27:38.755715 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22bcf4f-165b-408c-a1b8-d3f39086f4f3 admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:27:38.760301 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de2b029-3143-4bb5-ae72-a7ac31d132fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:27:38.762796 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de2b029-3143-4bb5-ae72-a7ac31d132fe admin admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:27:38.762796 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de2b029-3143-4bb5-ae72-a7ac31d132fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:27:38.762796 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de2b029-3143-4bb5-ae72-a7ac31d132fe admin admin] PATH_INFO: `/v3/roles` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:27:38.762796 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de2b029-3143-4bb5-ae72-a7ac31d132fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:27:38.763794 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de2b029-3143-4bb5-ae72-a7ac31d132fe admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:27:38.775068 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1147/2292] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:27:38 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:27:38.779351 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0885ef-34ee-44d5-a8ba-52d6a7eb852f admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:27:38.784635 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:27:38.790473 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0885ef-34ee-44d5-a8ba-52d6a7eb852f admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:27:38.797038 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2942e19a-89cb-4faf-9b00-94b8318eb13f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:27:38.797727 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2942e19a-89cb-4faf-9b00-94b8318eb13f admin admin] REQUEST_METHOD: `PUT` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:27:38.798042 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2942e19a-89cb-4faf-9b00-94b8318eb13f admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:27:38.798312 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2942e19a-89cb-4faf-9b00-94b8318eb13f admin admin] PATH_INFO: `/v3/projects/42aae54f98f24546b50c0b22972e63f4/users/8fb28c1e694843349ee45bce58589ccd/roles/d009411496b8452b88e77ffb8db015a7` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:27:38.811838 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de2b029-3143-4bb5-ae72-a7ac31d132fe admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:27:38.814218 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2942e19a-89cb-4faf-9b00-94b8318eb13f admin admin] RBAC: Authorizing `identity:create_grant(project_id=42aae54f98f24546b50c0b22972e63f4, user_id=8fb28c1e694843349ee45bce58589ccd, role_id=d009411496b8452b88e77ffb8db015a7)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:27:38.814483 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-4de2b029-3143-4bb5-ae72-a7ac31d132fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:27:38.817464 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2942e19a-89cb-4faf-9b00-94b8318eb13f admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:27:38.856604 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:27:38.901803 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de2b029-3143-4bb5-ae72-a7ac31d132fe admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:27:38.907498 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694f5734-f3e9-4825-9e5a-3ea737e0a4e3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8744c9de803048f583d0317d01f6a540', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:27:38.908108 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694f5734-f3e9-4825-9e5a-3ea737e0a4e3 service nova] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:27:38.908393 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694f5734-f3e9-4825-9e5a-3ea737e0a4e3 service nova] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:27:38.908671 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694f5734-f3e9-4825-9e5a-3ea737e0a4e3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:27:38.910433 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-694f5734-f3e9-4825-9e5a-3ea737e0a4e3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:27:38.966593 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694f5734-f3e9-4825-9e5a-3ea737e0a4e3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:27:38.968241 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694f5734-f3e9-4825-9e5a-3ea737e0a4e3 service nova] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:27:38.977783 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1148/2293] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:27:38 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 01:27:39.037789 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1146/2294] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 01:27:38 2022] PUT /identity/v3/projects/42aae54f98f24546b50c0b22972e63f4/users/8fb28c1e694843349ee45bce58589ccd/roles/d009411496b8452b88e77ffb8db015a7 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:27:39.041920 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694f5734-f3e9-4825-9e5a-3ea737e0a4e3 service nova] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:27:39.054325 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694f5734-f3e9-4825-9e5a-3ea737e0a4e3 service nova] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:27:39.111859 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70ff2a4-b1a4-4f5a-8dbe-7cde8ac16681 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:27:39.112640 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70ff2a4-b1a4-4f5a-8dbe-7cde8ac16681 admin admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:27:39.112867 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70ff2a4-b1a4-4f5a-8dbe-7cde8ac16681 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:27:39.113144 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70ff2a4-b1a4-4f5a-8dbe-7cde8ac16681 admin admin] PATH_INFO: `/v3/roles` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:27:39.113809 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70ff2a4-b1a4-4f5a-8dbe-7cde8ac16681 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:27:39.115481 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70ff2a4-b1a4-4f5a-8dbe-7cde8ac16681 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:27:39.125215 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1149/2295] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:27:39 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:27:39.130022 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2942e19a-89cb-4faf-9b00-94b8318eb13f admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:27:39.141793 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2942e19a-89cb-4faf-9b00-94b8318eb13f admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:27:39.147057 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3af152e-f3b1-436b-a9c0-96c16df08d2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:27:39.147662 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3af152e-f3b1-436b-a9c0-96c16df08d2e admin admin] REQUEST_METHOD: `PUT` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:27:39.147946 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3af152e-f3b1-436b-a9c0-96c16df08d2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:27:39.148223 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3af152e-f3b1-436b-a9c0-96c16df08d2e admin admin] PATH_INFO: `/v3/projects/42aae54f98f24546b50c0b22972e63f4/users/8fb28c1e694843349ee45bce58589ccd/roles/d009411496b8452b88e77ffb8db015a7` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:27:39.150128 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3af152e-f3b1-436b-a9c0-96c16df08d2e admin admin] RBAC: Authorizing `identity:create_grant(project_id=42aae54f98f24546b50c0b22972e63f4, user_id=8fb28c1e694843349ee45bce58589ccd, role_id=d009411496b8452b88e77ffb8db015a7)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:27:39.152790 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3af152e-f3b1-436b-a9c0-96c16df08d2e admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:27:39.164771 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1147/2296] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 01:27:39 2022] PUT /identity/v3/projects/42aae54f98f24546b50c0b22972e63f4/users/8fb28c1e694843349ee45bce58589ccd/roles/d009411496b8452b88e77ffb8db015a7 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:27:39.174994 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b99ff6-dc5e-4996-adce-bd71b50f717e None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:27:39.175283 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b99ff6-dc5e-4996-adce-bd71b50f717e None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:27:39.175678 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b99ff6-dc5e-4996-adce-bd71b50f717e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:27:39.203707 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-e5b99ff6-dc5e-4996-adce-bd71b50f717e None None] MFA Rules not processed for user `8fb28c1e694843349ee45bce58589ccd`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:27:39.205201 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-e5b99ff6-dc5e-4996-adce-bd71b50f717e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:27:39.273364 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1150/2297] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:27:39 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:27:39.284595 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3af152e-f3b1-436b-a9c0-96c16df08d2e admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:27:39.286357 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-a3af152e-f3b1-436b-a9c0-96c16df08d2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:27:39.342718 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3af152e-f3b1-436b-a9c0-96c16df08d2e admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:27:39.346463 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7563ab4-44fb-4d2a-9663-41ae76d99957 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5c8fc69c518412ba1aad61a9f4f558c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:27:39.348069 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7563ab4-44fb-4d2a-9663-41ae76d99957 service cinder] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:27:39.348069 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7563ab4-44fb-4d2a-9663-41ae76d99957 service cinder] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:27:39.348069 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7563ab4-44fb-4d2a-9663-41ae76d99957 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:27:39.356490 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7563ab4-44fb-4d2a-9663-41ae76d99957 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:27:39.357980 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7563ab4-44fb-4d2a-9663-41ae76d99957 service cinder] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:27:39.367850 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1148/2298] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:27:39 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:27:39.688418 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:27:40.504946 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b99ff6-dc5e-4996-adce-bd71b50f717e None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:27:40.570611 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b99ff6-dc5e-4996-adce-bd71b50f717e None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:27:40.576001 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961dd530-c0b1-4869-92c3-c19d13988b6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:27:40.576641 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961dd530-c0b1-4869-92c3-c19d13988b6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:27:40.576959 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961dd530-c0b1-4869-92c3-c19d13988b6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:27:40.577241 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961dd530-c0b1-4869-92c3-c19d13988b6c admin admin] PATH_INFO: `/v3/users/d7a3e2d79c1d4d44bd88471e4c18d58f` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:27:40.578436 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961dd530-c0b1-4869-92c3-c19d13988b6c admin admin] RBAC: Authorizing `identity:delete_user(user_id=d7a3e2d79c1d4d44bd88471e4c18d58f)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:27:40.580576 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961dd530-c0b1-4869-92c3-c19d13988b6c admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:27:40.672580 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7563ab4-44fb-4d2a-9663-41ae76d99957 service cinder] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:27:40.762281 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7563ab4-44fb-4d2a-9663-41ae76d99957 service cinder] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:27:40.768287 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d101c847-c477-474e-b6e8-34cc86bd34db service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8744c9de803048f583d0317d01f6a540', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:27:40.768955 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d101c847-c477-474e-b6e8-34cc86bd34db service nova] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:27:40.769101 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-961dd530-c0b1-4869-92c3-c19d13988b6c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd7a3e2d79c1d4d44bd88471e4c18d58f'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:27:40.769586 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d101c847-c477-474e-b6e8-34cc86bd34db service nova] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:27:40.769586 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d101c847-c477-474e-b6e8-34cc86bd34db service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:27:40.772229 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-d101c847-c477-474e-b6e8-34cc86bd34db service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:27:40.785745 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-961dd530-c0b1-4869-92c3-c19d13988b6c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd7a3e2d79c1d4d44bd88471e4c18d58f'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:27:40.787144 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-961dd530-c0b1-4869-92c3-c19d13988b6c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd7a3e2d79c1d4d44bd88471e4c18d58f'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:27:40.796104 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-961dd530-c0b1-4869-92c3-c19d13988b6c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd7a3e2d79c1d4d44bd88471e4c18d58f'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:27:40.839065 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d101c847-c477-474e-b6e8-34cc86bd34db service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:27:40.842157 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d101c847-c477-474e-b6e8-34cc86bd34db service nova] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:27:40.852953 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1149/2299] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:27:40 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:27:40.854043 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d101c847-c477-474e-b6e8-34cc86bd34db service nova] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:27:40.860126 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:27:40.865823 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-961dd530-c0b1-4869-92c3-c19d13988b6c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd7a3e2d79c1d4d44bd88471e4c18d58f'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:27:40.887603 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1151/2300] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 01:27:40 2022] DELETE /identity/v3/users/d7a3e2d79c1d4d44bd88471e4c18d58f => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:27:40.892214 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde38678-7c3a-4f33-89c7-3496bf8f6fbf None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:27:40.892540 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde38678-7c3a-4f33-89c7-3496bf8f6fbf None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:27:40.892858 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde38678-7c3a-4f33-89c7-3496bf8f6fbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:27:40.919236 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-bde38678-7c3a-4f33-89c7-3496bf8f6fbf None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:27:40.920478 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-bde38678-7c3a-4f33-89c7-3496bf8f6fbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:27:40.936884 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d101c847-c477-474e-b6e8-34cc86bd34db service nova] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:27:40.998626 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f553b2-ec05-4196-9bdc-7edbc7ee1b81 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0817c3320d5d41d8bba0550c8ae8dd60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:27:40.999487 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f553b2-ec05-4196-9bdc-7edbc7ee1b81 service neutron] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:27:41.000059 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f553b2-ec05-4196-9bdc-7edbc7ee1b81 service neutron] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:27:41.000522 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f553b2-ec05-4196-9bdc-7edbc7ee1b81 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:27:41.002768 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-37f553b2-ec05-4196-9bdc-7edbc7ee1b81 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:27:41.013772 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f553b2-ec05-4196-9bdc-7edbc7ee1b81 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:27:41.024353 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f553b2-ec05-4196-9bdc-7edbc7ee1b81 service neutron] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:27:41.025304 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1152/2301] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:27:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 01:27:41.025424 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1150/2302] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:27:40 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:27:41.028332 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde38678-7c3a-4f33-89c7-3496bf8f6fbf None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:27:41.030330 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-bde38678-7c3a-4f33-89c7-3496bf8f6fbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:27:41.044245 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde38678-7c3a-4f33-89c7-3496bf8f6fbf None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:27:41.050514 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3315693-caa7-4849-bb90-00356b84809f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0817c3320d5d41d8bba0550c8ae8dd60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:27:41.051683 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3315693-caa7-4849-bb90-00356b84809f service neutron] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:27:41.051968 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3315693-caa7-4849-bb90-00356b84809f service neutron] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:27:41.052207 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3315693-caa7-4849-bb90-00356b84809f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:27:41.066225 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:27:41.071037 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3315693-caa7-4849-bb90-00356b84809f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:27:41.072545 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3315693-caa7-4849-bb90-00356b84809f service neutron] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:27:41.085976 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1153/2303] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:27:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 01:27:41.884284 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f553b2-ec05-4196-9bdc-7edbc7ee1b81 service neutron] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:27:41.885880 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-37f553b2-ec05-4196-9bdc-7edbc7ee1b81 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:27:41.920618 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f553b2-ec05-4196-9bdc-7edbc7ee1b81 service neutron] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:27:41.925943 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffd5e0e-261b-48c5-9b2d-04ed6696c2c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:27:41.926628 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffd5e0e-261b-48c5-9b2d-04ed6696c2c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:27:41.926915 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffd5e0e-261b-48c5-9b2d-04ed6696c2c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:27:41.927199 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffd5e0e-261b-48c5-9b2d-04ed6696c2c2 admin admin] PATH_INFO: `/v3/projects/9ac94cad39a5413eabcc9b00306f9f16` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:27:41.928594 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffd5e0e-261b-48c5-9b2d-04ed6696c2c2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9ac94cad39a5413eabcc9b00306f9f16)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:27:41.931030 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffd5e0e-261b-48c5-9b2d-04ed6696c2c2 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:27:42.094509 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-7ffd5e0e-261b-48c5-9b2d-04ed6696c2c2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9ac94cad39a5413eabcc9b00306f9f16', 'request_id': 'req-7ffd5e0e-261b-48c5-9b2d-04ed6696c2c2'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:27:42.100034 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-7ffd5e0e-261b-48c5-9b2d-04ed6696c2c2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9ac94cad39a5413eabcc9b00306f9f16', 'request_id': 'req-7ffd5e0e-261b-48c5-9b2d-04ed6696c2c2'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:27:42.108258 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-7ffd5e0e-261b-48c5-9b2d-04ed6696c2c2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9ac94cad39a5413eabcc9b00306f9f16', 'request_id': 'req-7ffd5e0e-261b-48c5-9b2d-04ed6696c2c2'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:27:42.259038 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-7ffd5e0e-261b-48c5-9b2d-04ed6696c2c2 admin admin] The token cache is being invalidate because project 9ac94cad39a5413eabcc9b00306f9f16 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81749) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 01:27:42.259335 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-7ffd5e0e-261b-48c5-9b2d-04ed6696c2c2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:27:42.259994 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-7ffd5e0e-261b-48c5-9b2d-04ed6696c2c2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:27:42.261774 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1151/2304] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 01:27:41 2022] DELETE /identity/v3/projects/9ac94cad39a5413eabcc9b00306f9f16 => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:27:42.265780 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3315693-caa7-4849-bb90-00356b84809f service neutron] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:27:42.276803 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3315693-caa7-4849-bb90-00356b84809f service neutron] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:27:42.346446 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c501a3ba-392d-4f4e-a8f3-7060df5b9ba6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:27:42.346446 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c501a3ba-392d-4f4e-a8f3-7060df5b9ba6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:27:42.346446 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c501a3ba-392d-4f4e-a8f3-7060df5b9ba6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:27:42.346446 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c501a3ba-392d-4f4e-a8f3-7060df5b9ba6 admin admin] PATH_INFO: `/v3/users/8fb28c1e694843349ee45bce58589ccd` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:27:42.347039 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c501a3ba-392d-4f4e-a8f3-7060df5b9ba6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8fb28c1e694843349ee45bce58589ccd)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:27:42.349575 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c501a3ba-392d-4f4e-a8f3-7060df5b9ba6 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:27:42.529571 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-c501a3ba-392d-4f4e-a8f3-7060df5b9ba6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8fb28c1e694843349ee45bce58589ccd'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:27:42.551078 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-c501a3ba-392d-4f4e-a8f3-7060df5b9ba6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8fb28c1e694843349ee45bce58589ccd'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:27:42.551919 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-c501a3ba-392d-4f4e-a8f3-7060df5b9ba6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8fb28c1e694843349ee45bce58589ccd'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:27:42.560590 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-c501a3ba-392d-4f4e-a8f3-7060df5b9ba6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8fb28c1e694843349ee45bce58589ccd'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:27:42.630197 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-c501a3ba-392d-4f4e-a8f3-7060df5b9ba6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8fb28c1e694843349ee45bce58589ccd'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:27:42.640274 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1154/2305] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 01:27:42 2022] DELETE /identity/v3/users/8fb28c1e694843349ee45bce58589ccd => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:27:43.296635 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffd5e0e-261b-48c5-9b2d-04ed6696c2c2 admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:27:43.298329 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-7ffd5e0e-261b-48c5-9b2d-04ed6696c2c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:27:43.318173 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffd5e0e-261b-48c5-9b2d-04ed6696c2c2 admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:27:43.365339 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:27:43.378415 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c24e575-ccfa-463b-af7b-8ab2dbd9ada2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:27:43.379065 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c24e575-ccfa-463b-af7b-8ab2dbd9ada2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:27:43.379913 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c24e575-ccfa-463b-af7b-8ab2dbd9ada2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:27:43.380194 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c24e575-ccfa-463b-af7b-8ab2dbd9ada2 admin admin] PATH_INFO: `/v3/projects/42aae54f98f24546b50c0b22972e63f4` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:27:43.381358 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c24e575-ccfa-463b-af7b-8ab2dbd9ada2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=42aae54f98f24546b50c0b22972e63f4)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:27:43.383312 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c24e575-ccfa-463b-af7b-8ab2dbd9ada2 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:27:43.527595 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-1c24e575-ccfa-463b-af7b-8ab2dbd9ada2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '42aae54f98f24546b50c0b22972e63f4', 'request_id': 'req-1c24e575-ccfa-463b-af7b-8ab2dbd9ada2'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:27:43.532985 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-1c24e575-ccfa-463b-af7b-8ab2dbd9ada2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '42aae54f98f24546b50c0b22972e63f4', 'request_id': 'req-1c24e575-ccfa-463b-af7b-8ab2dbd9ada2'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:27:43.539586 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-1c24e575-ccfa-463b-af7b-8ab2dbd9ada2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '42aae54f98f24546b50c0b22972e63f4', 'request_id': 'req-1c24e575-ccfa-463b-af7b-8ab2dbd9ada2'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:27:43.710004 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-1c24e575-ccfa-463b-af7b-8ab2dbd9ada2 admin admin] The token cache is being invalidate because project 42aae54f98f24546b50c0b22972e63f4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81749) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 01:27:43.710357 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-1c24e575-ccfa-463b-af7b-8ab2dbd9ada2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:27:43.710925 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-1c24e575-ccfa-463b-af7b-8ab2dbd9ada2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:27:43.719725 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1152/2306] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 01:27:43 2022] DELETE /identity/v3/projects/42aae54f98f24546b50c0b22972e63f4 => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:27:43.719864 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ca8df0-e37b-498e-80ad-7f74f2777c21 None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:27:43.719864 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ca8df0-e37b-498e-80ad-7f74f2777c21 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:27:43.719864 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ca8df0-e37b-498e-80ad-7f74f2777c21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:27:43.738218 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-f2ca8df0-e37b-498e-80ad-7f74f2777c21 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:27:43.739988 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-f2ca8df0-e37b-498e-80ad-7f74f2777c21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:27:43.799367 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1155/2307] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:27:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:27:43.807734 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de38a12b-d1d3-4925-bb6d-29a58ea788b5 None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:27:43.808096 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de38a12b-d1d3-4925-bb6d-29a58ea788b5 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:27:43.808499 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de38a12b-d1d3-4925-bb6d-29a58ea788b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:27:43.829474 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-de38a12b-d1d3-4925-bb6d-29a58ea788b5 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:27:43.830751 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-de38a12b-d1d3-4925-bb6d-29a58ea788b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:27:43.845528 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1153/2308] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:27:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:27:43.850311 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ca8df0-e37b-498e-80ad-7f74f2777c21 None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:27:43.864561 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ca8df0-e37b-498e-80ad-7f74f2777c21 None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:27:43.870063 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b2f73d-0335-4b3a-a2a6-bb52880784bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:27:43.870789 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b2f73d-0335-4b3a-a2a6-bb52880784bd admin admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:27:43.871136 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b2f73d-0335-4b3a-a2a6-bb52880784bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:27:43.871482 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b2f73d-0335-4b3a-a2a6-bb52880784bd admin admin] PATH_INFO: `/v3/domains` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:27:43.872197 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b2f73d-0335-4b3a-a2a6-bb52880784bd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:27:43.872563 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b2f73d-0335-4b3a-a2a6-bb52880784bd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:27:43.874223 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b2f73d-0335-4b3a-a2a6-bb52880784bd admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:27:43.883155 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:27:43.886700 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1156/2309] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Aug 31 01:27:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:27:43.891173 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de38a12b-d1d3-4925-bb6d-29a58ea788b5 None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:27:43.911224 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de38a12b-d1d3-4925-bb6d-29a58ea788b5 None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:27:43.916133 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a306ab-15b5-4a02-91b6-83ef2f824fe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:27:43.917002 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a306ab-15b5-4a02-91b6-83ef2f824fe7 admin admin] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:27:43.917145 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a306ab-15b5-4a02-91b6-83ef2f824fe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:27:43.917310 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a306ab-15b5-4a02-91b6-83ef2f824fe7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:27:43.917952 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a306ab-15b5-4a02-91b6-83ef2f824fe7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:27:43.919873 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a306ab-15b5-4a02-91b6-83ef2f824fe7 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:27:44.023321 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1154/2310] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 01:27:43 2022] POST /identity/v3/projects => generated 385 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:27:44.024762 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b2f73d-0335-4b3a-a2a6-bb52880784bd admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:27:44.035281 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b2f73d-0335-4b3a-a2a6-bb52880784bd admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:27:44.104780 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f119723e-c11e-45c7-bb7e-25427bd054a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:27:44.105398 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f119723e-c11e-45c7-bb7e-25427bd054a7 admin admin] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:27:44.105705 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f119723e-c11e-45c7-bb7e-25427bd054a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:27:44.105977 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f119723e-c11e-45c7-bb7e-25427bd054a7 admin admin] PATH_INFO: `/v3/users` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:27:44.106605 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f119723e-c11e-45c7-bb7e-25427bd054a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:27:44.108511 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f119723e-c11e-45c7-bb7e-25427bd054a7 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:27:44.227497 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1157/2311] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 01:27:44 2022] POST /identity/v3/users => generated 293 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:27:44.231734 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a306ab-15b5-4a02-91b6-83ef2f824fe7 admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:27:44.242233 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a306ab-15b5-4a02-91b6-83ef2f824fe7 admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:27:44.246929 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066bc324-e213-42e6-84df-60e312f0ea7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:27:44.247550 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066bc324-e213-42e6-84df-60e312f0ea7c admin admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:27:44.247833 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066bc324-e213-42e6-84df-60e312f0ea7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:27:44.248126 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066bc324-e213-42e6-84df-60e312f0ea7c admin admin] PATH_INFO: `/v3/roles` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:27:44.248866 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066bc324-e213-42e6-84df-60e312f0ea7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:27:44.254815 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066bc324-e213-42e6-84df-60e312f0ea7c admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:27:44.261972 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1155/2312] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:27:44 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:27:44.266318 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f119723e-c11e-45c7-bb7e-25427bd054a7 admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:27:44.278423 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f119723e-c11e-45c7-bb7e-25427bd054a7 admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:27:44.283819 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bea6e1-6629-4b15-a79a-bb9f7a47c6e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:27:44.284438 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bea6e1-6629-4b15-a79a-bb9f7a47c6e6 admin admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:27:44.284745 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bea6e1-6629-4b15-a79a-bb9f7a47c6e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:27:44.285033 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bea6e1-6629-4b15-a79a-bb9f7a47c6e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:27:44.286137 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bea6e1-6629-4b15-a79a-bb9f7a47c6e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:27:44.287510 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bea6e1-6629-4b15-a79a-bb9f7a47c6e6 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:27:44.300761 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1158/2313] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:27:44 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:27:44.303639 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066bc324-e213-42e6-84df-60e312f0ea7c admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:27:44.314397 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066bc324-e213-42e6-84df-60e312f0ea7c admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:27:44.320527 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec847b6-55aa-4412-9b8c-5fb8725f345d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:27:44.321431 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec847b6-55aa-4412-9b8c-5fb8725f345d admin admin] REQUEST_METHOD: `PUT` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:27:44.321723 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec847b6-55aa-4412-9b8c-5fb8725f345d admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:27:44.322150 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec847b6-55aa-4412-9b8c-5fb8725f345d admin admin] PATH_INFO: `/v3/projects/47bddd5f136b4eefb4db32b9db5202e8/users/11907acb5dcc42e3820d9c269e5f9467/roles/d0b9768f4d014766b9326387f2e11f34` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:27:44.339980 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec847b6-55aa-4412-9b8c-5fb8725f345d admin admin] RBAC: Authorizing `identity:create_grant(project_id=47bddd5f136b4eefb4db32b9db5202e8, user_id=11907acb5dcc42e3820d9c269e5f9467, role_id=d0b9768f4d014766b9326387f2e11f34)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:27:44.342885 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec847b6-55aa-4412-9b8c-5fb8725f345d admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:27:44.453418 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1156/2314] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 01:27:44 2022] PUT /identity/v3/projects/47bddd5f136b4eefb4db32b9db5202e8/users/11907acb5dcc42e3820d9c269e5f9467/roles/d0b9768f4d014766b9326387f2e11f34 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:27:44.466066 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb9a622-f8ea-4401-8cfc-7e4d6d739294 None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:27:44.466339 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb9a622-f8ea-4401-8cfc-7e4d6d739294 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:27:44.466614 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb9a622-f8ea-4401-8cfc-7e4d6d739294 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:27:44.496548 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-ceb9a622-f8ea-4401-8cfc-7e4d6d739294 None None] MFA Rules not processed for user `11907acb5dcc42e3820d9c269e5f9467`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:27:44.497817 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-ceb9a622-f8ea-4401-8cfc-7e4d6d739294 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:27:44.563954 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1159/2315] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:27:44 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:27:44.568977 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec847b6-55aa-4412-9b8c-5fb8725f345d admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:27:44.634191 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec847b6-55aa-4412-9b8c-5fb8725f345d admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:27:44.638855 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e62b44-12d7-4607-8790-68c5cb37cc1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:27:44.639459 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e62b44-12d7-4607-8790-68c5cb37cc1b admin admin] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:27:44.639756 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e62b44-12d7-4607-8790-68c5cb37cc1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:27:44.640012 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e62b44-12d7-4607-8790-68c5cb37cc1b admin admin] PATH_INFO: `/v3/projects` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:27:44.640635 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e62b44-12d7-4607-8790-68c5cb37cc1b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:27:44.642725 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e62b44-12d7-4607-8790-68c5cb37cc1b admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:27:44.757454 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1157/2316] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 01:27:44 2022] POST /identity/v3/projects => generated 387 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:27:44.761498 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb9a622-f8ea-4401-8cfc-7e4d6d739294 None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:27:44.762183 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:27:44.772665 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb9a622-f8ea-4401-8cfc-7e4d6d739294 None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:27:44.829856 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e31cde5-4d30-4362-a0d8-a45380d95690 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:27:44.830462 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e31cde5-4d30-4362-a0d8-a45380d95690 admin admin] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:27:44.830749 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e31cde5-4d30-4362-a0d8-a45380d95690 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:27:44.831022 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e31cde5-4d30-4362-a0d8-a45380d95690 admin admin] PATH_INFO: `/v3/users` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:27:44.831661 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e31cde5-4d30-4362-a0d8-a45380d95690 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:27:44.833640 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e31cde5-4d30-4362-a0d8-a45380d95690 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:27:44.939422 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:27:44.945766 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1160/2317] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 01:27:44 2022] POST /identity/v3/users => generated 300 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:27:44.949854 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e62b44-12d7-4607-8790-68c5cb37cc1b admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:27:44.960969 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e62b44-12d7-4607-8790-68c5cb37cc1b admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:27:44.965658 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089af8c7-e0e6-4b10-a78e-77c05797ba19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:27:44.966339 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089af8c7-e0e6-4b10-a78e-77c05797ba19 admin admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:27:44.966625 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089af8c7-e0e6-4b10-a78e-77c05797ba19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:27:44.966905 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089af8c7-e0e6-4b10-a78e-77c05797ba19 admin admin] PATH_INFO: `/v3/roles` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:27:44.967598 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089af8c7-e0e6-4b10-a78e-77c05797ba19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:27:44.969263 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089af8c7-e0e6-4b10-a78e-77c05797ba19 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:27:44.980101 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1158/2318] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:27:44 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:27:44.983916 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e31cde5-4d30-4362-a0d8-a45380d95690 admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:27:44.994886 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e31cde5-4d30-4362-a0d8-a45380d95690 admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:27:44.999505 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6ee19b-4362-4814-a390-415397cf1562 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:27:45.000241 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6ee19b-4362-4814-a390-415397cf1562 admin admin] REQUEST_METHOD: `PUT` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:27:45.000549 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6ee19b-4362-4814-a390-415397cf1562 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:27:45.000892 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6ee19b-4362-4814-a390-415397cf1562 admin admin] PATH_INFO: `/v3/projects/798b96c6b15d40beb9033bbec97693eb/users/f301c1485fb44a2c9d66ce081c7adf06/roles/d009411496b8452b88e77ffb8db015a7` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:27:45.016245 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6ee19b-4362-4814-a390-415397cf1562 admin admin] RBAC: Authorizing `identity:create_grant(project_id=798b96c6b15d40beb9033bbec97693eb, user_id=f301c1485fb44a2c9d66ce081c7adf06, role_id=d009411496b8452b88e77ffb8db015a7)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:27:45.019081 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6ee19b-4362-4814-a390-415397cf1562 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:27:45.099617 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1161/2319] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 01:27:44 2022] PUT /identity/v3/projects/798b96c6b15d40beb9033bbec97693eb/users/f301c1485fb44a2c9d66ce081c7adf06/roles/d009411496b8452b88e77ffb8db015a7 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:27:45.103965 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089af8c7-e0e6-4b10-a78e-77c05797ba19 admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:27:45.115108 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089af8c7-e0e6-4b10-a78e-77c05797ba19 admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:27:45.179167 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0d532a-4321-4ec3-9505-ff82355365d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:27:45.179782 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0d532a-4321-4ec3-9505-ff82355365d8 admin admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:27:45.180067 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0d532a-4321-4ec3-9505-ff82355365d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:27:45.180347 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0d532a-4321-4ec3-9505-ff82355365d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:27:45.181020 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0d532a-4321-4ec3-9505-ff82355365d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:27:45.182677 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0d532a-4321-4ec3-9505-ff82355365d8 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:27:45.192660 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1159/2320] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:27:45 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:27:45.197001 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6ee19b-4362-4814-a390-415397cf1562 admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:27:45.207731 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6ee19b-4362-4814-a390-415397cf1562 admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:27:45.213124 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f489cd-eb76-4c7d-a4a8-2e028f5e32b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:27:45.213816 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f489cd-eb76-4c7d-a4a8-2e028f5e32b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:27:45.214100 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f489cd-eb76-4c7d-a4a8-2e028f5e32b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:27:45.214381 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f489cd-eb76-4c7d-a4a8-2e028f5e32b0 admin admin] PATH_INFO: `/v3/projects/798b96c6b15d40beb9033bbec97693eb/users/f301c1485fb44a2c9d66ce081c7adf06/roles/d009411496b8452b88e77ffb8db015a7` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:27:45.216315 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f489cd-eb76-4c7d-a4a8-2e028f5e32b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=798b96c6b15d40beb9033bbec97693eb, user_id=f301c1485fb44a2c9d66ce081c7adf06, role_id=d009411496b8452b88e77ffb8db015a7)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:27:45.218995 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f489cd-eb76-4c7d-a4a8-2e028f5e32b0 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:27:45.230034 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1162/2321] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 01:27:45 2022] PUT /identity/v3/projects/798b96c6b15d40beb9033bbec97693eb/users/f301c1485fb44a2c9d66ce081c7adf06/roles/d009411496b8452b88e77ffb8db015a7 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:27:45.244092 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec607178-1edd-4ba7-a4cf-7e04cda20edf None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:27:45.244092 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec607178-1edd-4ba7-a4cf-7e04cda20edf None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:27:45.244092 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec607178-1edd-4ba7-a4cf-7e04cda20edf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:27:45.270681 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-ec607178-1edd-4ba7-a4cf-7e04cda20edf None None] MFA Rules not processed for user `f301c1485fb44a2c9d66ce081c7adf06`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:27:45.278259 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-ec607178-1edd-4ba7-a4cf-7e04cda20edf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:27:45.344192 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1160/2322] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:27:45 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:27:45.355724 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f489cd-eb76-4c7d-a4a8-2e028f5e32b0 admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:27:45.357455 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-b6f489cd-eb76-4c7d-a4a8-2e028f5e32b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:27:45.438568 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f489cd-eb76-4c7d-a4a8-2e028f5e32b0 admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:27:45.444492 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bed23b-c93d-471e-8be5-6f5c089abd0b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5c8fc69c518412ba1aad61a9f4f558c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:27:45.445565 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bed23b-c93d-471e-8be5-6f5c089abd0b service cinder] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:27:45.446064 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bed23b-c93d-471e-8be5-6f5c089abd0b service cinder] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:27:45.446565 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bed23b-c93d-471e-8be5-6f5c089abd0b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:27:45.463278 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bed23b-c93d-471e-8be5-6f5c089abd0b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:27:45.466131 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bed23b-c93d-471e-8be5-6f5c089abd0b service cinder] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:27:45.484116 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1163/2323] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:27:45 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:27:45.515012 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec607178-1edd-4ba7-a4cf-7e04cda20edf None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:27:45.587400 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec607178-1edd-4ba7-a4cf-7e04cda20edf None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:27:45.594541 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74de050-2b80-491f-bd23-f11be821aaf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:27:45.594541 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74de050-2b80-491f-bd23-f11be821aaf9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:27:45.594785 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74de050-2b80-491f-bd23-f11be821aaf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:27:45.595040 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74de050-2b80-491f-bd23-f11be821aaf9 admin admin] PATH_INFO: `/v3/users/11907acb5dcc42e3820d9c269e5f9467` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:27:45.596209 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74de050-2b80-491f-bd23-f11be821aaf9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=11907acb5dcc42e3820d9c269e5f9467)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:27:45.598795 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74de050-2b80-491f-bd23-f11be821aaf9 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:27:45.610065 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:27:45.810691 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-b74de050-2b80-491f-bd23-f11be821aaf9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '11907acb5dcc42e3820d9c269e5f9467'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:27:45.886997 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-b74de050-2b80-491f-bd23-f11be821aaf9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '11907acb5dcc42e3820d9c269e5f9467'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:27:45.887707 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-b74de050-2b80-491f-bd23-f11be821aaf9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '11907acb5dcc42e3820d9c269e5f9467'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:27:45.896904 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-b74de050-2b80-491f-bd23-f11be821aaf9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '11907acb5dcc42e3820d9c269e5f9467'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:27:45.896904 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-b74de050-2b80-491f-bd23-f11be821aaf9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '11907acb5dcc42e3820d9c269e5f9467'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:27:45.918443 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1161/2324] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 01:27:45 2022] DELETE /identity/v3/users/11907acb5dcc42e3820d9c269e5f9467 => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:27:45.923445 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32b6610-65b6-46e2-9f5c-f9a32a8326d0 None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:27:45.923928 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32b6610-65b6-46e2-9f5c-f9a32a8326d0 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:27:45.924036 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32b6610-65b6-46e2-9f5c-f9a32a8326d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:27:45.943953 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-e32b6610-65b6-46e2-9f5c-f9a32a8326d0 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:27:45.945214 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-e32b6610-65b6-46e2-9f5c-f9a32a8326d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:27:46.005206 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1164/2325] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:27:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:27:46.014779 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74de050-2b80-491f-bd23-f11be821aaf9 admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:27:46.016188 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-b74de050-2b80-491f-bd23-f11be821aaf9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:27:46.075667 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74de050-2b80-491f-bd23-f11be821aaf9 admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:27:46.079497 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e73a8d-1953-486a-9d87-6664bad3c98f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0817c3320d5d41d8bba0550c8ae8dd60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:27:46.080423 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e73a8d-1953-486a-9d87-6664bad3c98f service neutron] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:27:46.080610 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e73a8d-1953-486a-9d87-6664bad3c98f service neutron] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:27:46.080922 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e73a8d-1953-486a-9d87-6664bad3c98f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:27:46.090907 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e73a8d-1953-486a-9d87-6664bad3c98f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:27:46.092344 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e73a8d-1953-486a-9d87-6664bad3c98f service neutron] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:27:46.102068 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1162/2326] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:27:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:27:46.770271 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32b6610-65b6-46e2-9f5c-f9a32a8326d0 None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:27:46.775115 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-e32b6610-65b6-46e2-9f5c-f9a32a8326d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:27:46.796188 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32b6610-65b6-46e2-9f5c-f9a32a8326d0 None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:27:46.811574 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996997e7-726a-4417-94e5-13a65ae50ea2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:27:46.812242 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996997e7-726a-4417-94e5-13a65ae50ea2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:27:46.812529 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996997e7-726a-4417-94e5-13a65ae50ea2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:27:46.812934 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996997e7-726a-4417-94e5-13a65ae50ea2 admin admin] PATH_INFO: `/v3/projects/47bddd5f136b4eefb4db32b9db5202e8` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:27:46.814152 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996997e7-726a-4417-94e5-13a65ae50ea2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=47bddd5f136b4eefb4db32b9db5202e8)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:27:46.816167 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996997e7-726a-4417-94e5-13a65ae50ea2 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:27:46.926625 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-996997e7-726a-4417-94e5-13a65ae50ea2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '47bddd5f136b4eefb4db32b9db5202e8', 'request_id': 'req-996997e7-726a-4417-94e5-13a65ae50ea2'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:27:46.933086 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-996997e7-726a-4417-94e5-13a65ae50ea2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '47bddd5f136b4eefb4db32b9db5202e8', 'request_id': 'req-996997e7-726a-4417-94e5-13a65ae50ea2'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:27:46.935203 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=92,keystone:INSERT=2,keystone:DELETE=6 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:27:47.001601 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-996997e7-726a-4417-94e5-13a65ae50ea2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '47bddd5f136b4eefb4db32b9db5202e8', 'request_id': 'req-996997e7-726a-4417-94e5-13a65ae50ea2'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:27:47.015436 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-996997e7-726a-4417-94e5-13a65ae50ea2 admin admin] The token cache is being invalidate because project 47bddd5f136b4eefb4db32b9db5202e8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81750) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 01:27:47.015775 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-996997e7-726a-4417-94e5-13a65ae50ea2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:27:47.016434 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-996997e7-726a-4417-94e5-13a65ae50ea2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:27:47.018071 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1165/2327] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 01:27:46 2022] DELETE /identity/v3/projects/47bddd5f136b4eefb4db32b9db5202e8 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:27:47.023260 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e73a8d-1953-486a-9d87-6664bad3c98f service neutron] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:27:47.034064 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e73a8d-1953-486a-9d87-6664bad3c98f service neutron] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:27:47.100389 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245050dc-84af-4466-b608-e061b456265f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:27:47.101074 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245050dc-84af-4466-b608-e061b456265f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:27:47.101546 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245050dc-84af-4466-b608-e061b456265f admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:27:47.101888 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245050dc-84af-4466-b608-e061b456265f admin admin] PATH_INFO: `/v3/users/f301c1485fb44a2c9d66ce081c7adf06` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:27:47.103107 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245050dc-84af-4466-b608-e061b456265f admin admin] RBAC: Authorizing `identity:delete_user(user_id=f301c1485fb44a2c9d66ce081c7adf06)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:27:47.105190 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245050dc-84af-4466-b608-e061b456265f admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:27:47.368755 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:DELETE=15,keystone:SELECT=84,keystone:INSERT=1 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:27:47.381497 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-245050dc-84af-4466-b608-e061b456265f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f301c1485fb44a2c9d66ce081c7adf06'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:27:47.470528 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-245050dc-84af-4466-b608-e061b456265f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f301c1485fb44a2c9d66ce081c7adf06'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:27:47.471160 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-245050dc-84af-4466-b608-e061b456265f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f301c1485fb44a2c9d66ce081c7adf06'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:27:47.479071 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-245050dc-84af-4466-b608-e061b456265f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f301c1485fb44a2c9d66ce081c7adf06'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:27:47.479759 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-245050dc-84af-4466-b608-e061b456265f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f301c1485fb44a2c9d66ce081c7adf06'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:27:47.498256 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1163/2328] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 01:27:47 2022] DELETE /identity/v3/users/f301c1485fb44a2c9d66ce081c7adf06 => generated 0 bytes in 476 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 01:27:47.992180 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996997e7-726a-4417-94e5-13a65ae50ea2 admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:27:47.993479 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-996997e7-726a-4417-94e5-13a65ae50ea2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:27:48.007140 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996997e7-726a-4417-94e5-13a65ae50ea2 admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:27:48.062967 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abbc792-9926-42a2-99d6-1a432935865d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:27:48.063593 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abbc792-9926-42a2-99d6-1a432935865d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:27:48.063876 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abbc792-9926-42a2-99d6-1a432935865d admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:27:48.064146 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abbc792-9926-42a2-99d6-1a432935865d admin admin] PATH_INFO: `/v3/projects/798b96c6b15d40beb9033bbec97693eb` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:27:48.065270 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abbc792-9926-42a2-99d6-1a432935865d admin admin] RBAC: Authorizing `identity:delete_project(project_id=798b96c6b15d40beb9033bbec97693eb)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:27:48.067184 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abbc792-9926-42a2-99d6-1a432935865d admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:27:48.180310 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-6abbc792-9926-42a2-99d6-1a432935865d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '798b96c6b15d40beb9033bbec97693eb', 'request_id': 'req-6abbc792-9926-42a2-99d6-1a432935865d'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:27:48.185945 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-6abbc792-9926-42a2-99d6-1a432935865d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '798b96c6b15d40beb9033bbec97693eb', 'request_id': 'req-6abbc792-9926-42a2-99d6-1a432935865d'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:27:48.252457 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-6abbc792-9926-42a2-99d6-1a432935865d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '798b96c6b15d40beb9033bbec97693eb', 'request_id': 'req-6abbc792-9926-42a2-99d6-1a432935865d'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:27:48.266396 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-6abbc792-9926-42a2-99d6-1a432935865d admin admin] The token cache is being invalidate because project 798b96c6b15d40beb9033bbec97693eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81750) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 01:27:48.266695 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-6abbc792-9926-42a2-99d6-1a432935865d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:27:48.267346 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-6abbc792-9926-42a2-99d6-1a432935865d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:27:48.268869 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1166/2329] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 01:27:47 2022] DELETE /identity/v3/projects/798b96c6b15d40beb9033bbec97693eb => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:27:48.273719 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe2f77d-800e-461a-bbd1-e1aa70a43951 None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:27:48.274193 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe2f77d-800e-461a-bbd1-e1aa70a43951 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:27:48.274299 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe2f77d-800e-461a-bbd1-e1aa70a43951 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:27:48.295853 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-8fe2f77d-800e-461a-bbd1-e1aa70a43951 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:27:48.295853 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-8fe2f77d-800e-461a-bbd1-e1aa70a43951 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:27:48.357860 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1164/2330] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:27:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:27:48.366309 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c291cb4-7642-4a00-b23b-f93779d863ca None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:27:48.366678 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c291cb4-7642-4a00-b23b-f93779d863ca None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:27:48.367024 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c291cb4-7642-4a00-b23b-f93779d863ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:27:48.387126 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-3c291cb4-7642-4a00-b23b-f93779d863ca None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:27:48.388351 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-3c291cb4-7642-4a00-b23b-f93779d863ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:27:48.397552 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1167/2331] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:27:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:27:48.407404 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe2f77d-800e-461a-bbd1-e1aa70a43951 None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:27:48.418500 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe2f77d-800e-461a-bbd1-e1aa70a43951 None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:27:48.423941 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a78f8b-fa0d-46dd-83cc-c2501fd20dc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:27:48.424562 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a78f8b-fa0d-46dd-83cc-c2501fd20dc5 admin admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:27:48.424884 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a78f8b-fa0d-46dd-83cc-c2501fd20dc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:27:48.425167 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a78f8b-fa0d-46dd-83cc-c2501fd20dc5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:27:48.425853 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a78f8b-fa0d-46dd-83cc-c2501fd20dc5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:27:48.426164 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a78f8b-fa0d-46dd-83cc-c2501fd20dc5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:27:48.427821 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a78f8b-fa0d-46dd-83cc-c2501fd20dc5 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:27:48.440773 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1165/2332] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Aug 31 01:27:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:27:48.444402 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c291cb4-7642-4a00-b23b-f93779d863ca None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:27:48.455768 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c291cb4-7642-4a00-b23b-f93779d863ca None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:27:48.461635 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b5c61d-c252-4d88-a7cf-b2e9174d12a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:27:48.462323 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b5c61d-c252-4d88-a7cf-b2e9174d12a7 admin admin] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:27:48.462601 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b5c61d-c252-4d88-a7cf-b2e9174d12a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:27:48.462870 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b5c61d-c252-4d88-a7cf-b2e9174d12a7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:27:48.463512 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b5c61d-c252-4d88-a7cf-b2e9174d12a7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:27:48.465442 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b5c61d-c252-4d88-a7cf-b2e9174d12a7 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:27:48.607201 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1168/2333] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 01:27:48 2022] POST /identity/v3/projects => generated 383 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:27:48.608320 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a78f8b-fa0d-46dd-83cc-c2501fd20dc5 admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:27:48.618848 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a78f8b-fa0d-46dd-83cc-c2501fd20dc5 admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:27:48.675459 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5586e1d9-530d-4191-99ee-c530b2172840 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:27:48.675958 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5586e1d9-530d-4191-99ee-c530b2172840 admin admin] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:27:48.676239 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5586e1d9-530d-4191-99ee-c530b2172840 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:27:48.676513 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5586e1d9-530d-4191-99ee-c530b2172840 admin admin] PATH_INFO: `/v3/users` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:27:48.677196 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5586e1d9-530d-4191-99ee-c530b2172840 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:27:48.679153 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5586e1d9-530d-4191-99ee-c530b2172840 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:27:48.775581 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1166/2334] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 01:27:48 2022] POST /identity/v3/users => generated 292 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 01:27:48.776881 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b5c61d-c252-4d88-a7cf-b2e9174d12a7 admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:27:48.787779 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b5c61d-c252-4d88-a7cf-b2e9174d12a7 admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:27:48.792661 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cce916f-c547-4695-a9b0-3a8ff4b02367 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:27:48.793518 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cce916f-c547-4695-a9b0-3a8ff4b02367 admin admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:27:48.795711 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cce916f-c547-4695-a9b0-3a8ff4b02367 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:27:48.795711 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cce916f-c547-4695-a9b0-3a8ff4b02367 admin admin] PATH_INFO: `/v3/roles` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:27:48.795711 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cce916f-c547-4695-a9b0-3a8ff4b02367 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:27:48.796633 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cce916f-c547-4695-a9b0-3a8ff4b02367 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:27:48.807458 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1169/2335] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:27:48 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:27:48.811524 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5586e1d9-530d-4191-99ee-c530b2172840 admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:27:48.823002 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5586e1d9-530d-4191-99ee-c530b2172840 admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:27:48.827881 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bb112c-6fbd-45fc-9ed4-1d81bf3e4026 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:27:48.830030 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bb112c-6fbd-45fc-9ed4-1d81bf3e4026 admin admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:27:48.830030 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bb112c-6fbd-45fc-9ed4-1d81bf3e4026 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:27:48.830030 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bb112c-6fbd-45fc-9ed4-1d81bf3e4026 admin admin] PATH_INFO: `/v3/roles` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:27:48.830308 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bb112c-6fbd-45fc-9ed4-1d81bf3e4026 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:27:48.831889 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bb112c-6fbd-45fc-9ed4-1d81bf3e4026 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:27:48.844292 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1167/2336] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:27:48 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:27:48.848204 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cce916f-c547-4695-a9b0-3a8ff4b02367 admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:27:48.861742 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cce916f-c547-4695-a9b0-3a8ff4b02367 admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:27:48.866683 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e1174f-2834-4c63-8fae-27a1f851b2e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:27:48.867414 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e1174f-2834-4c63-8fae-27a1f851b2e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:27:48.867702 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e1174f-2834-4c63-8fae-27a1f851b2e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:27:48.867977 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e1174f-2834-4c63-8fae-27a1f851b2e1 admin admin] PATH_INFO: `/v3/projects/60b9b33aeefe4c0ba34bd24091384be3/users/20af7f9689534dc08b99fdea569e47e1/roles/d0b9768f4d014766b9326387f2e11f34` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:27:48.883913 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e1174f-2834-4c63-8fae-27a1f851b2e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=60b9b33aeefe4c0ba34bd24091384be3, user_id=20af7f9689534dc08b99fdea569e47e1, role_id=d0b9768f4d014766b9326387f2e11f34)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:27:48.886797 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e1174f-2834-4c63-8fae-27a1f851b2e1 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:27:48.956154 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1170/2337] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 01:27:48 2022] PUT /identity/v3/projects/60b9b33aeefe4c0ba34bd24091384be3/users/20af7f9689534dc08b99fdea569e47e1/roles/d0b9768f4d014766b9326387f2e11f34 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:27:48.967902 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2364854e-6660-4701-97ee-9346f80d6495 None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:27:48.968193 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2364854e-6660-4701-97ee-9346f80d6495 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:27:48.968473 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2364854e-6660-4701-97ee-9346f80d6495 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:27:48.997300 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-2364854e-6660-4701-97ee-9346f80d6495 None None] MFA Rules not processed for user `20af7f9689534dc08b99fdea569e47e1`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:27:48.998506 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-2364854e-6660-4701-97ee-9346f80d6495 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:27:49.002436 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:27:49.064563 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1168/2338] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:27:48 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:27:49.069685 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e1174f-2834-4c63-8fae-27a1f851b2e1 admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:27:49.083899 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:INSERT=4,keystone:DELETE=8,keystone:SELECT=88 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:27:49.132425 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e1174f-2834-4c63-8fae-27a1f851b2e1 admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:27:49.137141 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cad767-59a8-4159-bf92-5aaab3ea8434 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:27:49.137752 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cad767-59a8-4159-bf92-5aaab3ea8434 admin admin] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:27:49.138027 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cad767-59a8-4159-bf92-5aaab3ea8434 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:27:49.138308 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cad767-59a8-4159-bf92-5aaab3ea8434 admin admin] PATH_INFO: `/v3/projects` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:27:49.138989 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cad767-59a8-4159-bf92-5aaab3ea8434 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:27:49.141070 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cad767-59a8-4159-bf92-5aaab3ea8434 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:27:49.328800 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1171/2339] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 01:27:49 2022] POST /identity/v3/projects => generated 383 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:27:49.333031 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2364854e-6660-4701-97ee-9346f80d6495 None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:27:49.343796 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2364854e-6660-4701-97ee-9346f80d6495 None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:27:49.399633 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387e0e02-c787-4eab-9b0f-644479d2e199 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:27:49.400234 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387e0e02-c787-4eab-9b0f-644479d2e199 admin admin] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:27:49.400515 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387e0e02-c787-4eab-9b0f-644479d2e199 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:27:49.400817 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387e0e02-c787-4eab-9b0f-644479d2e199 admin admin] PATH_INFO: `/v3/users` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:27:49.401442 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387e0e02-c787-4eab-9b0f-644479d2e199 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:27:49.403426 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387e0e02-c787-4eab-9b0f-644479d2e199 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:27:49.515806 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1169/2340] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 01:27:49 2022] POST /identity/v3/users => generated 292 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:27:49.519849 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cad767-59a8-4159-bf92-5aaab3ea8434 admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:27:49.530301 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cad767-59a8-4159-bf92-5aaab3ea8434 admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:27:49.535024 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a7e645-b761-4efb-9cf4-cd2b54752f63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:27:49.536128 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a7e645-b761-4efb-9cf4-cd2b54752f63 admin admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:27:49.536457 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a7e645-b761-4efb-9cf4-cd2b54752f63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:27:49.536674 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a7e645-b761-4efb-9cf4-cd2b54752f63 admin admin] PATH_INFO: `/v3/roles` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:27:49.537402 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a7e645-b761-4efb-9cf4-cd2b54752f63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:27:49.538996 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a7e645-b761-4efb-9cf4-cd2b54752f63 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:27:49.549503 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1172/2341] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:27:49 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 01:27:49.554233 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387e0e02-c787-4eab-9b0f-644479d2e199 admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:27:49.564390 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387e0e02-c787-4eab-9b0f-644479d2e199 admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:27:49.568983 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a133a589-86fd-42e0-a456-86045e5837a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:27:49.569576 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a133a589-86fd-42e0-a456-86045e5837a0 admin admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:27:49.569854 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a133a589-86fd-42e0-a456-86045e5837a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:27:49.570125 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a133a589-86fd-42e0-a456-86045e5837a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:27:49.570807 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a133a589-86fd-42e0-a456-86045e5837a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:27:49.572395 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a133a589-86fd-42e0-a456-86045e5837a0 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:27:49.582314 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1170/2342] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:27:49 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:27:49.586465 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a7e645-b761-4efb-9cf4-cd2b54752f63 admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:27:49.596359 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a7e645-b761-4efb-9cf4-cd2b54752f63 admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:27:49.601637 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2c03e9-3909-4b2e-b474-92c71859fe37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:27:49.602300 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2c03e9-3909-4b2e-b474-92c71859fe37 admin admin] REQUEST_METHOD: `PUT` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:27:49.602583 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2c03e9-3909-4b2e-b474-92c71859fe37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:27:49.602860 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2c03e9-3909-4b2e-b474-92c71859fe37 admin admin] PATH_INFO: `/v3/projects/e5c617095f334ff5aeca76f38a149d28/users/97e55ffc15fd41ee99eefc53a90f96dd/roles/d0b9768f4d014766b9326387f2e11f34` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:27:49.618290 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2c03e9-3909-4b2e-b474-92c71859fe37 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e5c617095f334ff5aeca76f38a149d28, user_id=97e55ffc15fd41ee99eefc53a90f96dd, role_id=d0b9768f4d014766b9326387f2e11f34)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:27:49.620947 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2c03e9-3909-4b2e-b474-92c71859fe37 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:27:49.698559 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1173/2343] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 01:27:49 2022] PUT /identity/v3/projects/e5c617095f334ff5aeca76f38a149d28/users/97e55ffc15fd41ee99eefc53a90f96dd/roles/d0b9768f4d014766b9326387f2e11f34 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:27:49.709705 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60f0b00-c7ff-4bfc-be15-598be37857f6 None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:27:49.709999 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60f0b00-c7ff-4bfc-be15-598be37857f6 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:27:49.710282 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60f0b00-c7ff-4bfc-be15-598be37857f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:27:49.739564 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-c60f0b00-c7ff-4bfc-be15-598be37857f6 None None] MFA Rules not processed for user `97e55ffc15fd41ee99eefc53a90f96dd`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:27:49.740820 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-c60f0b00-c7ff-4bfc-be15-598be37857f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:27:49.804416 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1171/2344] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:27:49 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 01:27:49.809188 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2c03e9-3909-4b2e-b474-92c71859fe37 admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:27:49.872043 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2c03e9-3909-4b2e-b474-92c71859fe37 admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:27:49.876666 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09adde84-78c2-4143-9bba-6f358ec416b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:27:49.877294 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09adde84-78c2-4143-9bba-6f358ec416b3 admin admin] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:27:49.877579 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09adde84-78c2-4143-9bba-6f358ec416b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:27:49.877857 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09adde84-78c2-4143-9bba-6f358ec416b3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:27:49.878494 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09adde84-78c2-4143-9bba-6f358ec416b3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:27:49.880457 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09adde84-78c2-4143-9bba-6f358ec416b3 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:27:49.983321 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1174/2345] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 01:27:49 2022] POST /identity/v3/projects => generated 383 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:27:49.984830 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60f0b00-c7ff-4bfc-be15-598be37857f6 None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:27:49.996287 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60f0b00-c7ff-4bfc-be15-598be37857f6 None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:27:50.007312 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:27:50.057578 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8324916-ba2f-4726-924e-75794fdcf6b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:27:50.058191 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8324916-ba2f-4726-924e-75794fdcf6b3 admin admin] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:27:50.058469 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8324916-ba2f-4726-924e-75794fdcf6b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:27:50.058743 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8324916-ba2f-4726-924e-75794fdcf6b3 admin admin] PATH_INFO: `/v3/users` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:27:50.059406 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8324916-ba2f-4726-924e-75794fdcf6b3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:27:50.061394 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8324916-ba2f-4726-924e-75794fdcf6b3 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:27:50.181875 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1172/2346] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 01:27:49 2022] POST /identity/v3/users => generated 298 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:27:50.183404 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09adde84-78c2-4143-9bba-6f358ec416b3 admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:27:50.193986 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09adde84-78c2-4143-9bba-6f358ec416b3 admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:27:50.198912 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ba58ea-a398-4e1a-aa56-7379de4d66ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:27:50.199519 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ba58ea-a398-4e1a-aa56-7379de4d66ff admin admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:27:50.199801 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ba58ea-a398-4e1a-aa56-7379de4d66ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:27:50.200092 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ba58ea-a398-4e1a-aa56-7379de4d66ff admin admin] PATH_INFO: `/v3/roles` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:27:50.200787 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ba58ea-a398-4e1a-aa56-7379de4d66ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:27:50.202624 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ba58ea-a398-4e1a-aa56-7379de4d66ff admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:27:50.214126 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1175/2347] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:27:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:27:50.218903 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8324916-ba2f-4726-924e-75794fdcf6b3 admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:27:50.229429 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8324916-ba2f-4726-924e-75794fdcf6b3 admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:27:50.233960 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3bebd0-ab2d-45b5-8d31-f054997126b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:27:50.234626 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3bebd0-ab2d-45b5-8d31-f054997126b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:27:50.234910 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3bebd0-ab2d-45b5-8d31-f054997126b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:27:50.235186 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3bebd0-ab2d-45b5-8d31-f054997126b2 admin admin] PATH_INFO: `/v3/projects/d7c1ad07a02e4376ba12a5458a279483/users/1c941ecb016f45bcb88003d7688b5f95/roles/d009411496b8452b88e77ffb8db015a7` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:27:50.250809 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3bebd0-ab2d-45b5-8d31-f054997126b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7c1ad07a02e4376ba12a5458a279483, user_id=1c941ecb016f45bcb88003d7688b5f95, role_id=d009411496b8452b88e77ffb8db015a7)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:27:50.253632 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3bebd0-ab2d-45b5-8d31-f054997126b2 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:27:50.369075 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1173/2348] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 01:27:50 2022] PUT /identity/v3/projects/d7c1ad07a02e4376ba12a5458a279483/users/1c941ecb016f45bcb88003d7688b5f95/roles/d009411496b8452b88e77ffb8db015a7 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:27:50.373009 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ba58ea-a398-4e1a-aa56-7379de4d66ff admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:27:50.384396 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ba58ea-a398-4e1a-aa56-7379de4d66ff admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:27:50.421966 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:27:50.445777 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e503e280-f9ff-4cd5-8d6e-0c8ce351824c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:27:50.446399 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e503e280-f9ff-4cd5-8d6e-0c8ce351824c admin admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:27:50.446684 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e503e280-f9ff-4cd5-8d6e-0c8ce351824c admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:27:50.446958 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e503e280-f9ff-4cd5-8d6e-0c8ce351824c admin admin] PATH_INFO: `/v3/roles` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:27:50.447609 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e503e280-f9ff-4cd5-8d6e-0c8ce351824c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:27:50.449275 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e503e280-f9ff-4cd5-8d6e-0c8ce351824c admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:27:50.461005 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1176/2349] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:27:50 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:27:50.465647 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3bebd0-ab2d-45b5-8d31-f054997126b2 admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:27:50.477801 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3bebd0-ab2d-45b5-8d31-f054997126b2 admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:27:50.482830 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6342cfb1-75d2-400e-9af8-f8a331743574 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:27:50.483944 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6342cfb1-75d2-400e-9af8-f8a331743574 admin admin] REQUEST_METHOD: `PUT` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:27:50.484967 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6342cfb1-75d2-400e-9af8-f8a331743574 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:27:50.485383 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6342cfb1-75d2-400e-9af8-f8a331743574 admin admin] PATH_INFO: `/v3/projects/d7c1ad07a02e4376ba12a5458a279483/users/1c941ecb016f45bcb88003d7688b5f95/roles/d009411496b8452b88e77ffb8db015a7` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:27:50.487567 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6342cfb1-75d2-400e-9af8-f8a331743574 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7c1ad07a02e4376ba12a5458a279483, user_id=1c941ecb016f45bcb88003d7688b5f95, role_id=d009411496b8452b88e77ffb8db015a7)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:27:50.490541 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6342cfb1-75d2-400e-9af8-f8a331743574 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:27:50.501779 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1174/2350] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 01:27:50 2022] PUT /identity/v3/projects/d7c1ad07a02e4376ba12a5458a279483/users/1c941ecb016f45bcb88003d7688b5f95/roles/d009411496b8452b88e77ffb8db015a7 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:27:50.513247 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc35dde9-b705-4adc-9992-21c97418ad20 None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:27:50.513537 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc35dde9-b705-4adc-9992-21c97418ad20 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:27:50.513813 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc35dde9-b705-4adc-9992-21c97418ad20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:27:50.542974 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-dc35dde9-b705-4adc-9992-21c97418ad20 None None] MFA Rules not processed for user `1c941ecb016f45bcb88003d7688b5f95`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:27:50.544236 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-dc35dde9-b705-4adc-9992-21c97418ad20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:27:50.615206 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1177/2351] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:27:50 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:27:50.626888 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6342cfb1-75d2-400e-9af8-f8a331743574 admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:27:50.629550 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-6342cfb1-75d2-400e-9af8-f8a331743574 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:27:50.686535 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6342cfb1-75d2-400e-9af8-f8a331743574 admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:27:50.690168 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c538f0f9-d4dd-4f16-ac40-917c898b3d1e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5c8fc69c518412ba1aad61a9f4f558c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:27:50.690825 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c538f0f9-d4dd-4f16-ac40-917c898b3d1e service cinder] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:27:50.691055 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c538f0f9-d4dd-4f16-ac40-917c898b3d1e service cinder] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:27:50.691355 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c538f0f9-d4dd-4f16-ac40-917c898b3d1e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:27:50.747197 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c538f0f9-d4dd-4f16-ac40-917c898b3d1e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:27:50.748677 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c538f0f9-d4dd-4f16-ac40-917c898b3d1e service cinder] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:27:50.758150 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1175/2352] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:27:50 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:27:52.692253 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc35dde9-b705-4adc-9992-21c97418ad20 None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:27:52.693741 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-dc35dde9-b705-4adc-9992-21c97418ad20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:27:52.706161 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc35dde9-b705-4adc-9992-21c97418ad20 None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:27:52.710629 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa5bb67-31ee-4a5c-9ad2-6b9eaf896fb2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5c8fc69c518412ba1aad61a9f4f558c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:27:52.711307 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa5bb67-31ee-4a5c-9ad2-6b9eaf896fb2 service cinder] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:27:52.711666 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa5bb67-31ee-4a5c-9ad2-6b9eaf896fb2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:27:52.712034 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa5bb67-31ee-4a5c-9ad2-6b9eaf896fb2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:27:52.775160 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa5bb67-31ee-4a5c-9ad2-6b9eaf896fb2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:27:52.776953 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa5bb67-31ee-4a5c-9ad2-6b9eaf896fb2 service cinder] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:27:52.786881 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1178/2353] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:27:52 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:27:52.953429 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c538f0f9-d4dd-4f16-ac40-917c898b3d1e service cinder] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:27:52.955184 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-c538f0f9-d4dd-4f16-ac40-917c898b3d1e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:27:52.978382 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:27:53.031469 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c538f0f9-d4dd-4f16-ac40-917c898b3d1e service cinder] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:27:53.037744 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b730285-a3a1-41d1-8a16-07194eb040e1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8744c9de803048f583d0317d01f6a540', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:27:53.038364 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b730285-a3a1-41d1-8a16-07194eb040e1 service nova] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:27:53.038649 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b730285-a3a1-41d1-8a16-07194eb040e1 service nova] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:27:53.038924 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b730285-a3a1-41d1-8a16-07194eb040e1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:27:53.040651 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-4b730285-a3a1-41d1-8a16-07194eb040e1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:27:53.058034 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b730285-a3a1-41d1-8a16-07194eb040e1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:27:53.060592 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b730285-a3a1-41d1-8a16-07194eb040e1 service nova] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:27:53.072063 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1176/2354] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:27:52 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:27:54.222580 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa5bb67-31ee-4a5c-9ad2-6b9eaf896fb2 service cinder] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:27:54.231752 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa5bb67-31ee-4a5c-9ad2-6b9eaf896fb2 service cinder] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:27:54.235790 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23079c2-1cc1-4456-9816-caae0de9bbf8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5c8fc69c518412ba1aad61a9f4f558c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:27:54.236425 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23079c2-1cc1-4456-9816-caae0de9bbf8 service cinder] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:27:54.236962 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23079c2-1cc1-4456-9816-caae0de9bbf8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:27:54.237412 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23079c2-1cc1-4456-9816-caae0de9bbf8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:27:54.247053 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23079c2-1cc1-4456-9816-caae0de9bbf8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:27:54.248501 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23079c2-1cc1-4456-9816-caae0de9bbf8 service cinder] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:27:54.258382 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1179/2355] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:27:54 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:27:58.652466 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b730285-a3a1-41d1-8a16-07194eb040e1 service nova] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:27:58.713650 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b730285-a3a1-41d1-8a16-07194eb040e1 service nova] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:27:58.718148 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1df793-6c2c-41f7-b3ee-a9d46fdb9f68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:27:58.718764 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1df793-6c2c-41f7-b3ee-a9d46fdb9f68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:27:58.719043 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1df793-6c2c-41f7-b3ee-a9d46fdb9f68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:27:58.719313 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1df793-6c2c-41f7-b3ee-a9d46fdb9f68 admin admin] PATH_INFO: `/v3/users/20af7f9689534dc08b99fdea569e47e1` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:27:58.720481 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1df793-6c2c-41f7-b3ee-a9d46fdb9f68 admin admin] RBAC: Authorizing `identity:delete_user(user_id=20af7f9689534dc08b99fdea569e47e1)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:27:58.722448 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1df793-6c2c-41f7-b3ee-a9d46fdb9f68 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:27:58.837564 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-1c1df793-6c2c-41f7-b3ee-a9d46fdb9f68 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '20af7f9689534dc08b99fdea569e47e1'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:27:58.881068 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-1c1df793-6c2c-41f7-b3ee-a9d46fdb9f68 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '20af7f9689534dc08b99fdea569e47e1'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:27:58.881691 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-1c1df793-6c2c-41f7-b3ee-a9d46fdb9f68 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '20af7f9689534dc08b99fdea569e47e1'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:27:58.888777 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-1c1df793-6c2c-41f7-b3ee-a9d46fdb9f68 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '20af7f9689534dc08b99fdea569e47e1'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:27:58.889409 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-1c1df793-6c2c-41f7-b3ee-a9d46fdb9f68 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '20af7f9689534dc08b99fdea569e47e1'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:27:58.909850 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1177/2356] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 01:27:58 2022] DELETE /identity/v3/users/20af7f9689534dc08b99fdea569e47e1 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:27:58.914254 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84e1d28-f105-48cc-ad3d-58d7b30c7021 None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:27:58.914707 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84e1d28-f105-48cc-ad3d-58d7b30c7021 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:27:58.914806 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84e1d28-f105-48cc-ad3d-58d7b30c7021 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:27:58.934062 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-d84e1d28-f105-48cc-ad3d-58d7b30c7021 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:27:58.935267 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-d84e1d28-f105-48cc-ad3d-58d7b30c7021 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:27:58.974435 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:27:58.992630 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1180/2357] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:27:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 01:27:59.007022 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1df793-6c2c-41f7-b3ee-a9d46fdb9f68 admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:27:59.008440 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-1c1df793-6c2c-41f7-b3ee-a9d46fdb9f68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:27:59.036247 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:27:59.067183 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1df793-6c2c-41f7-b3ee-a9d46fdb9f68 admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:27:59.070874 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5257279-f1bd-4848-b623-ba5d776c23bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0817c3320d5d41d8bba0550c8ae8dd60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:27:59.071506 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5257279-f1bd-4848-b623-ba5d776c23bf service neutron] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:27:59.071783 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5257279-f1bd-4848-b623-ba5d776c23bf service neutron] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:27:59.073143 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5257279-f1bd-4848-b623-ba5d776c23bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:27:59.081221 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5257279-f1bd-4848-b623-ba5d776c23bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:27:59.082646 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5257279-f1bd-4848-b623-ba5d776c23bf service neutron] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:27:59.095347 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1178/2358] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:27:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:27:59.506414 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84e1d28-f105-48cc-ad3d-58d7b30c7021 None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:27:59.508048 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-d84e1d28-f105-48cc-ad3d-58d7b30c7021 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:27:59.535744 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84e1d28-f105-48cc-ad3d-58d7b30c7021 None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:27:59.541861 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158faaa7-28cc-46ae-822d-234a8360c3e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:27:59.542507 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158faaa7-28cc-46ae-822d-234a8360c3e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:27:59.542791 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158faaa7-28cc-46ae-822d-234a8360c3e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:27:59.543141 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158faaa7-28cc-46ae-822d-234a8360c3e6 admin admin] PATH_INFO: `/v3/projects/60b9b33aeefe4c0ba34bd24091384be3` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:27:59.544284 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158faaa7-28cc-46ae-822d-234a8360c3e6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=60b9b33aeefe4c0ba34bd24091384be3)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:27:59.546262 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158faaa7-28cc-46ae-822d-234a8360c3e6 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:27:59.630606 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-158faaa7-28cc-46ae-822d-234a8360c3e6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '60b9b33aeefe4c0ba34bd24091384be3', 'request_id': 'req-158faaa7-28cc-46ae-822d-234a8360c3e6'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:27:59.636041 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-158faaa7-28cc-46ae-822d-234a8360c3e6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '60b9b33aeefe4c0ba34bd24091384be3', 'request_id': 'req-158faaa7-28cc-46ae-822d-234a8360c3e6'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:27:59.698173 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-158faaa7-28cc-46ae-822d-234a8360c3e6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '60b9b33aeefe4c0ba34bd24091384be3', 'request_id': 'req-158faaa7-28cc-46ae-822d-234a8360c3e6'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:27:59.709886 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-158faaa7-28cc-46ae-822d-234a8360c3e6 admin admin] The token cache is being invalidate because project 60b9b33aeefe4c0ba34bd24091384be3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81750) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 01:27:59.710189 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-158faaa7-28cc-46ae-822d-234a8360c3e6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:27:59.710811 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-158faaa7-28cc-46ae-822d-234a8360c3e6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:27:59.712288 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1181/2359] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 01:27:59 2022] DELETE /identity/v3/projects/60b9b33aeefe4c0ba34bd24091384be3 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:27:59.716314 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5257279-f1bd-4848-b623-ba5d776c23bf service neutron] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:27:59.726159 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5257279-f1bd-4848-b623-ba5d776c23bf service neutron] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:27:59.782204 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d73e302-828a-4746-93ce-a0884d7d0188 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:27:59.782830 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d73e302-828a-4746-93ce-a0884d7d0188 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:27:59.783112 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d73e302-828a-4746-93ce-a0884d7d0188 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:27:59.783388 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d73e302-828a-4746-93ce-a0884d7d0188 admin admin] PATH_INFO: `/v3/users/97e55ffc15fd41ee99eefc53a90f96dd` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:27:59.784498 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d73e302-828a-4746-93ce-a0884d7d0188 admin admin] RBAC: Authorizing `identity:delete_user(user_id=97e55ffc15fd41ee99eefc53a90f96dd)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:27:59.786453 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d73e302-828a-4746-93ce-a0884d7d0188 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:27:59.985359 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-1d73e302-828a-4746-93ce-a0884d7d0188 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '97e55ffc15fd41ee99eefc53a90f96dd'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:28:00.069077 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-1d73e302-828a-4746-93ce-a0884d7d0188 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '97e55ffc15fd41ee99eefc53a90f96dd'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:28:00.069699 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-1d73e302-828a-4746-93ce-a0884d7d0188 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '97e55ffc15fd41ee99eefc53a90f96dd'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:28:00.076616 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-1d73e302-828a-4746-93ce-a0884d7d0188 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '97e55ffc15fd41ee99eefc53a90f96dd'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:28:00.077262 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-1d73e302-828a-4746-93ce-a0884d7d0188 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '97e55ffc15fd41ee99eefc53a90f96dd'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:28:00.094418 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1179/2360] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 01:27:59 2022] DELETE /identity/v3/users/97e55ffc15fd41ee99eefc53a90f96dd => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:28:00.466518 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158faaa7-28cc-46ae-822d-234a8360c3e6 admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:28:00.468077 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-158faaa7-28cc-46ae-822d-234a8360c3e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:28:00.481812 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158faaa7-28cc-46ae-822d-234a8360c3e6 admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:28:00.543037 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d887a3-7ff3-4942-83aa-f04701132bfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:28:00.543756 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d887a3-7ff3-4942-83aa-f04701132bfb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:28:00.544088 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d887a3-7ff3-4942-83aa-f04701132bfb admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:28:00.544362 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d887a3-7ff3-4942-83aa-f04701132bfb admin admin] PATH_INFO: `/v3/projects/e5c617095f334ff5aeca76f38a149d28` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:28:00.545574 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d887a3-7ff3-4942-83aa-f04701132bfb admin admin] RBAC: Authorizing `identity:delete_project(project_id=e5c617095f334ff5aeca76f38a149d28)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:28:00.547512 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d887a3-7ff3-4942-83aa-f04701132bfb admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:28:00.663715 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-b1d887a3-7ff3-4942-83aa-f04701132bfb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e5c617095f334ff5aeca76f38a149d28', 'request_id': 'req-b1d887a3-7ff3-4942-83aa-f04701132bfb'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:28:00.671602 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-b1d887a3-7ff3-4942-83aa-f04701132bfb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e5c617095f334ff5aeca76f38a149d28', 'request_id': 'req-b1d887a3-7ff3-4942-83aa-f04701132bfb'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:28:00.738811 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-b1d887a3-7ff3-4942-83aa-f04701132bfb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e5c617095f334ff5aeca76f38a149d28', 'request_id': 'req-b1d887a3-7ff3-4942-83aa-f04701132bfb'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:28:00.747501 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:28:00.752215 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-b1d887a3-7ff3-4942-83aa-f04701132bfb admin admin] The token cache is being invalidate because project e5c617095f334ff5aeca76f38a149d28 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81750) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 01:28:00.752508 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-b1d887a3-7ff3-4942-83aa-f04701132bfb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:28:00.753201 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-b1d887a3-7ff3-4942-83aa-f04701132bfb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:28:00.755923 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1182/2361] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 01:28:00 2022] DELETE /identity/v3/projects/e5c617095f334ff5aeca76f38a149d28 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:28:00.759601 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d73e302-828a-4746-93ce-a0884d7d0188 admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:28:00.761754 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-1d73e302-828a-4746-93ce-a0884d7d0188 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:28:00.774645 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d73e302-828a-4746-93ce-a0884d7d0188 admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:28:00.818533 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:28:00.830997 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123b632a-8f05-4a47-8d64-9b03a2614f6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:28:00.831669 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123b632a-8f05-4a47-8d64-9b03a2614f6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:28:00.831957 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123b632a-8f05-4a47-8d64-9b03a2614f6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:28:00.832238 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123b632a-8f05-4a47-8d64-9b03a2614f6a admin admin] PATH_INFO: `/v3/users/1c941ecb016f45bcb88003d7688b5f95` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:28:00.833403 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123b632a-8f05-4a47-8d64-9b03a2614f6a admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c941ecb016f45bcb88003d7688b5f95)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:28:00.837258 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123b632a-8f05-4a47-8d64-9b03a2614f6a admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:28:01.032360 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-123b632a-8f05-4a47-8d64-9b03a2614f6a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c941ecb016f45bcb88003d7688b5f95'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:28:01.103550 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-123b632a-8f05-4a47-8d64-9b03a2614f6a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c941ecb016f45bcb88003d7688b5f95'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:28:01.104158 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-123b632a-8f05-4a47-8d64-9b03a2614f6a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c941ecb016f45bcb88003d7688b5f95'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:28:01.111257 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-123b632a-8f05-4a47-8d64-9b03a2614f6a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c941ecb016f45bcb88003d7688b5f95'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:28:01.111878 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-123b632a-8f05-4a47-8d64-9b03a2614f6a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c941ecb016f45bcb88003d7688b5f95'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:28:01.129066 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1180/2362] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 01:28:00 2022] DELETE /identity/v3/users/1c941ecb016f45bcb88003d7688b5f95 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 01:28:01.523742 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d887a3-7ff3-4942-83aa-f04701132bfb admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:28:01.525266 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-b1d887a3-7ff3-4942-83aa-f04701132bfb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:28:01.539663 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d887a3-7ff3-4942-83aa-f04701132bfb admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:28:01.596864 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f3feb6-d5e4-4ab4-b0fe-86fa1c932d03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:28:01.597521 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f3feb6-d5e4-4ab4-b0fe-86fa1c932d03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:28:01.598079 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f3feb6-d5e4-4ab4-b0fe-86fa1c932d03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:28:01.598079 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f3feb6-d5e4-4ab4-b0fe-86fa1c932d03 admin admin] PATH_INFO: `/v3/projects/d7c1ad07a02e4376ba12a5458a279483` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:28:01.599186 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f3feb6-d5e4-4ab4-b0fe-86fa1c932d03 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d7c1ad07a02e4376ba12a5458a279483)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:28:01.601290 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f3feb6-d5e4-4ab4-b0fe-86fa1c932d03 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:28:01.694912 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-71f3feb6-d5e4-4ab4-b0fe-86fa1c932d03 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd7c1ad07a02e4376ba12a5458a279483', 'request_id': 'req-71f3feb6-d5e4-4ab4-b0fe-86fa1c932d03'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:28:01.700929 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-71f3feb6-d5e4-4ab4-b0fe-86fa1c932d03 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd7c1ad07a02e4376ba12a5458a279483', 'request_id': 'req-71f3feb6-d5e4-4ab4-b0fe-86fa1c932d03'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:28:01.779076 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-71f3feb6-d5e4-4ab4-b0fe-86fa1c932d03 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd7c1ad07a02e4376ba12a5458a279483', 'request_id': 'req-71f3feb6-d5e4-4ab4-b0fe-86fa1c932d03'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:28:01.790682 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-71f3feb6-d5e4-4ab4-b0fe-86fa1c932d03 admin admin] The token cache is being invalidate because project d7c1ad07a02e4376ba12a5458a279483 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81750) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 01:28:01.791278 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-71f3feb6-d5e4-4ab4-b0fe-86fa1c932d03 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:28:01.792000 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-71f3feb6-d5e4-4ab4-b0fe-86fa1c932d03 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:28:01.793808 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1183/2363] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 01:28:01 2022] DELETE /identity/v3/projects/d7c1ad07a02e4376ba12a5458a279483 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:28:01.800308 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e580343d-58a4-4646-81be-9a5849af7dd2 None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:28:01.800308 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e580343d-58a4-4646-81be-9a5849af7dd2 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:28:01.800308 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e580343d-58a4-4646-81be-9a5849af7dd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:28:01.820308 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-e580343d-58a4-4646-81be-9a5849af7dd2 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:28:01.821545 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-e580343d-58a4-4646-81be-9a5849af7dd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:28:01.888680 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1181/2364] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:28:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:28:01.896363 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75667ddf-3ed4-4a58-817a-ac49a5f9d9e7 None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:28:01.896652 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75667ddf-3ed4-4a58-817a-ac49a5f9d9e7 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:28:01.896958 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75667ddf-3ed4-4a58-817a-ac49a5f9d9e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:28:01.920666 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-75667ddf-3ed4-4a58-817a-ac49a5f9d9e7 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:28:01.922360 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-75667ddf-3ed4-4a58-817a-ac49a5f9d9e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:28:01.932773 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1184/2365] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:28:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:28:01.936135 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e580343d-58a4-4646-81be-9a5849af7dd2 None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:28:01.947536 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e580343d-58a4-4646-81be-9a5849af7dd2 None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:28:01.956074 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297d1242-e418-4b82-b276-3af067a41bea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:28:01.956849 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297d1242-e418-4b82-b276-3af067a41bea admin admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:28:01.957206 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297d1242-e418-4b82-b276-3af067a41bea admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:28:01.957554 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297d1242-e418-4b82-b276-3af067a41bea admin admin] PATH_INFO: `/v3/domains` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:28:01.958285 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297d1242-e418-4b82-b276-3af067a41bea admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:28:01.958662 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297d1242-e418-4b82-b276-3af067a41bea admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:28:01.960383 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297d1242-e418-4b82-b276-3af067a41bea admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:28:01.978003 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1182/2366] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Aug 31 01:28:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:28:01.979168 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75667ddf-3ed4-4a58-817a-ac49a5f9d9e7 None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:28:01.989039 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75667ddf-3ed4-4a58-817a-ac49a5f9d9e7 None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:28:01.994061 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd40066-8f1a-48d4-a949-6987482a6f9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:28:01.994655 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd40066-8f1a-48d4-a949-6987482a6f9e admin admin] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:28:01.994937 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd40066-8f1a-48d4-a949-6987482a6f9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:28:01.995211 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd40066-8f1a-48d4-a949-6987482a6f9e admin admin] PATH_INFO: `/v3/projects` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:28:01.995841 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd40066-8f1a-48d4-a949-6987482a6f9e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:28:01.997769 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd40066-8f1a-48d4-a949-6987482a6f9e admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:28:02.067526 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9260ed24-1e95-48c9-81be-cfdb514109fe None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:28:02.067856 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9260ed24-1e95-48c9-81be-cfdb514109fe None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:28:02.068095 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9260ed24-1e95-48c9-81be-cfdb514109fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:28:02.078767 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1185/2367] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 01:28:01 2022] POST /identity/v3/projects => generated 373 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:28:02.080119 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd40066-8f1a-48d4-a949-6987482a6f9e admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:28:02.090339 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-9260ed24-1e95-48c9-81be-cfdb514109fe None None] MFA Rules not processed for user `b3efb4f9daac4a0ca05a36dd361f17e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:28:02.091567 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-9260ed24-1e95-48c9-81be-cfdb514109fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:28:02.141500 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:28:02.144778 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd40066-8f1a-48d4-a949-6987482a6f9e admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:28:02.149464 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ed0795-0e30-4983-bd23-e6b60539ac03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:28:02.150188 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ed0795-0e30-4983-bd23-e6b60539ac03 admin admin] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:28:02.150489 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1183/2368] 10.222.0.20 () {58 vars in 1009 bytes} [Wed Aug 31 01:28:02 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:28:02.150638 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ed0795-0e30-4983-bd23-e6b60539ac03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:28:02.150895 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ed0795-0e30-4983-bd23-e6b60539ac03 admin admin] PATH_INFO: `/v3/users` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:28:02.151539 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ed0795-0e30-4983-bd23-e6b60539ac03 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:28:02.153565 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ed0795-0e30-4983-bd23-e6b60539ac03 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:28:02.162030 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:28:02.263519 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1186/2369] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 01:28:02 2022] POST /identity/v3/users => generated 287 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:28:02.267640 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9260ed24-1e95-48c9-81be-cfdb514109fe None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:28:02.279146 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9260ed24-1e95-48c9-81be-cfdb514109fe None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:28:02.284422 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf3dfd2-679a-48bd-98a6-22fd617993a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:28:02.285580 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf3dfd2-679a-48bd-98a6-22fd617993a2 admin admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:28:02.285860 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf3dfd2-679a-48bd-98a6-22fd617993a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:28:02.286117 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf3dfd2-679a-48bd-98a6-22fd617993a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:28:02.286759 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf3dfd2-679a-48bd-98a6-22fd617993a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:28:02.288380 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf3dfd2-679a-48bd-98a6-22fd617993a2 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:28:02.302645 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1184/2370] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:28:02 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:28:02.305037 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf3dfd2-679a-48bd-98a6-22fd617993a2 admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:28:02.305288 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ed0795-0e30-4983-bd23-e6b60539ac03 admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:28:02.306901 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-9cf3dfd2-679a-48bd-98a6-22fd617993a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:28:02.316807 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ed0795-0e30-4983-bd23-e6b60539ac03 admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:28:02.321325 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb348c5f-138b-409b-ab33-9bc994fd901d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:28:02.322051 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb348c5f-138b-409b-ab33-9bc994fd901d admin admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:28:02.322241 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb348c5f-138b-409b-ab33-9bc994fd901d admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:28:02.322485 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb348c5f-138b-409b-ab33-9bc994fd901d admin admin] PATH_INFO: `/v3/roles` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:28:02.323173 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb348c5f-138b-409b-ab33-9bc994fd901d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:28:02.324798 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb348c5f-138b-409b-ab33-9bc994fd901d admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:28:02.335668 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1187/2371] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:28:02 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:28:02.339510 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb348c5f-138b-409b-ab33-9bc994fd901d admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:28:02.350415 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb348c5f-138b-409b-ab33-9bc994fd901d admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:28:02.355908 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6ddfa7-df26-413e-8f34-5e20be7d74dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:28:02.356630 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6ddfa7-df26-413e-8f34-5e20be7d74dc admin admin] REQUEST_METHOD: `PUT` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:28:02.358959 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6ddfa7-df26-413e-8f34-5e20be7d74dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:28:02.358959 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6ddfa7-df26-413e-8f34-5e20be7d74dc admin admin] PATH_INFO: `/v3/projects/0f7f97ac08b346739b1aca4862ef2c22/users/f78047042f8c40f59b2fdc5f3d607140/roles/d0b9768f4d014766b9326387f2e11f34` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:28:02.376148 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6ddfa7-df26-413e-8f34-5e20be7d74dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f7f97ac08b346739b1aca4862ef2c22, user_id=f78047042f8c40f59b2fdc5f3d607140, role_id=d0b9768f4d014766b9326387f2e11f34)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:28:02.379072 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6ddfa7-df26-413e-8f34-5e20be7d74dc admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:28:02.386540 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf3dfd2-679a-48bd-98a6-22fd617993a2 admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:28:02.391844 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21014061-72f9-4e92-ae6c-198d92367f40 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc1d010919ec40878ae829cc8dceec4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:28:02.392585 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21014061-72f9-4e92-ae6c-198d92367f40 service swift] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:28:02.392892 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21014061-72f9-4e92-ae6c-198d92367f40 service swift] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:28:02.393169 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21014061-72f9-4e92-ae6c-198d92367f40 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:28:02.401875 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21014061-72f9-4e92-ae6c-198d92367f40 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:28:02.403541 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21014061-72f9-4e92-ae6c-198d92367f40 service swift] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:28:02.412818 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1185/2372] 10.222.0.20 () {60 vars in 1384 bytes} [Wed Aug 31 01:28:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 01:28:02.458892 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1188/2373] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 01:28:02 2022] PUT /identity/v3/projects/0f7f97ac08b346739b1aca4862ef2c22/users/f78047042f8c40f59b2fdc5f3d607140/roles/d0b9768f4d014766b9326387f2e11f34 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 01:28:02.461056 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77de267a-8e3e-48cd-864d-534c75f61293 None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:28:02.461344 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77de267a-8e3e-48cd-864d-534c75f61293 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:28:02.461677 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77de267a-8e3e-48cd-864d-534c75f61293 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:28:02.502656 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-77de267a-8e3e-48cd-864d-534c75f61293 None None] MFA Rules not processed for user `f78047042f8c40f59b2fdc5f3d607140`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:28:02.505458 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-77de267a-8e3e-48cd-864d-534c75f61293 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:28:02.599239 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1186/2374] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:28:02 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:28:02.611176 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6ddfa7-df26-413e-8f34-5e20be7d74dc admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:28:02.613999 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-8e6ddfa7-df26-413e-8f34-5e20be7d74dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:28:02.685208 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6ddfa7-df26-413e-8f34-5e20be7d74dc admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:28:02.689470 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e673a45a-fc04-430c-8766-1504343e8a00 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5c8fc69c518412ba1aad61a9f4f558c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:28:02.690093 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e673a45a-fc04-430c-8766-1504343e8a00 service cinder] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:28:02.690382 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e673a45a-fc04-430c-8766-1504343e8a00 service cinder] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:28:02.690663 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e673a45a-fc04-430c-8766-1504343e8a00 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:28:02.703056 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e673a45a-fc04-430c-8766-1504343e8a00 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:28:02.704809 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e673a45a-fc04-430c-8766-1504343e8a00 service cinder] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:28:02.715289 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1189/2375] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:28:02 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 01:28:02.747850 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77de267a-8e3e-48cd-864d-534c75f61293 None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:28:02.826374 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77de267a-8e3e-48cd-864d-534c75f61293 None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:28:02.836096 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affe0d69-4707-414d-b231-96cfbd0f981d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:28:02.836769 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affe0d69-4707-414d-b231-96cfbd0f981d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:28:02.837060 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affe0d69-4707-414d-b231-96cfbd0f981d admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:28:02.837339 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affe0d69-4707-414d-b231-96cfbd0f981d admin admin] PATH_INFO: `/v3/users/f78047042f8c40f59b2fdc5f3d607140` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:28:02.838513 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affe0d69-4707-414d-b231-96cfbd0f981d admin admin] RBAC: Authorizing `identity:delete_user(user_id=f78047042f8c40f59b2fdc5f3d607140)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:28:02.840448 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affe0d69-4707-414d-b231-96cfbd0f981d admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:28:02.856336 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:28:03.034421 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-affe0d69-4707-414d-b231-96cfbd0f981d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f78047042f8c40f59b2fdc5f3d607140'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:28:03.138593 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-affe0d69-4707-414d-b231-96cfbd0f981d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f78047042f8c40f59b2fdc5f3d607140'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:28:03.139235 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-affe0d69-4707-414d-b231-96cfbd0f981d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f78047042f8c40f59b2fdc5f3d607140'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:28:03.167865 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-affe0d69-4707-414d-b231-96cfbd0f981d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f78047042f8c40f59b2fdc5f3d607140'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:28:03.168641 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-affe0d69-4707-414d-b231-96cfbd0f981d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f78047042f8c40f59b2fdc5f3d607140'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:28:03.194688 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1187/2376] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 01:28:02 2022] DELETE /identity/v3/users/f78047042f8c40f59b2fdc5f3d607140 => generated 0 bytes in 445 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:28:03.196453 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb4184c-a145-4daf-bfcb-b48ff09adb6a None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:28:03.197015 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb4184c-a145-4daf-bfcb-b48ff09adb6a None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:28:03.197442 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb4184c-a145-4daf-bfcb-b48ff09adb6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:28:03.220693 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-5fb4184c-a145-4daf-bfcb-b48ff09adb6a None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:28:03.222457 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-5fb4184c-a145-4daf-bfcb-b48ff09adb6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:28:03.313759 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1190/2377] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:28:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:28:03.323494 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affe0d69-4707-414d-b231-96cfbd0f981d admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:28:03.325118 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-affe0d69-4707-414d-b231-96cfbd0f981d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:28:03.398006 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affe0d69-4707-414d-b231-96cfbd0f981d admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:28:03.402242 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988982f6-8897-45f2-8b91-3792a9a0fe53 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0817c3320d5d41d8bba0550c8ae8dd60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:28:03.402892 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988982f6-8897-45f2-8b91-3792a9a0fe53 service neutron] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:28:03.403178 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988982f6-8897-45f2-8b91-3792a9a0fe53 service neutron] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:28:03.403457 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988982f6-8897-45f2-8b91-3792a9a0fe53 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:28:03.413731 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988982f6-8897-45f2-8b91-3792a9a0fe53 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:28:03.415186 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988982f6-8897-45f2-8b91-3792a9a0fe53 service neutron] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:28:03.426935 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1188/2378] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:28:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:28:03.884071 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb4184c-a145-4daf-bfcb-b48ff09adb6a None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:28:03.886470 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-5fb4184c-a145-4daf-bfcb-b48ff09adb6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:28:03.926653 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb4184c-a145-4daf-bfcb-b48ff09adb6a None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:28:03.933767 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3784ce-1911-49d8-bf5f-c86eaf8011fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:28:03.934556 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3784ce-1911-49d8-bf5f-c86eaf8011fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:28:03.934915 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3784ce-1911-49d8-bf5f-c86eaf8011fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:28:03.935260 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3784ce-1911-49d8-bf5f-c86eaf8011fb admin admin] PATH_INFO: `/v3/projects/0f7f97ac08b346739b1aca4862ef2c22` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:28:03.936618 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3784ce-1911-49d8-bf5f-c86eaf8011fb admin admin] RBAC: Authorizing `identity:delete_project(project_id=0f7f97ac08b346739b1aca4862ef2c22)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:28:03.938778 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3784ce-1911-49d8-bf5f-c86eaf8011fb admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:28:04.048074 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-4c3784ce-1911-49d8-bf5f-c86eaf8011fb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0f7f97ac08b346739b1aca4862ef2c22', 'request_id': 'req-4c3784ce-1911-49d8-bf5f-c86eaf8011fb'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:28:04.050121 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:INSERT=4,keystone:SELECT=91,keystone:DELETE=5 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:28:04.057659 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-4c3784ce-1911-49d8-bf5f-c86eaf8011fb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0f7f97ac08b346739b1aca4862ef2c22', 'request_id': 'req-4c3784ce-1911-49d8-bf5f-c86eaf8011fb'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:28:04.115842 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-4c3784ce-1911-49d8-bf5f-c86eaf8011fb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0f7f97ac08b346739b1aca4862ef2c22', 'request_id': 'req-4c3784ce-1911-49d8-bf5f-c86eaf8011fb'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:28:04.129029 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-4c3784ce-1911-49d8-bf5f-c86eaf8011fb admin admin] The token cache is being invalidate because project 0f7f97ac08b346739b1aca4862ef2c22 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81750) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 01:28:04.129385 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-4c3784ce-1911-49d8-bf5f-c86eaf8011fb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:28:04.130041 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-4c3784ce-1911-49d8-bf5f-c86eaf8011fb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:28:04.131543 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1191/2379] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 01:28:03 2022] DELETE /identity/v3/projects/0f7f97ac08b346739b1aca4862ef2c22 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:28:04.138375 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c097e6-6cd4-4f13-90c9-406e3937b715 None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:28:04.141428 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c097e6-6cd4-4f13-90c9-406e3937b715 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:28:04.141428 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c097e6-6cd4-4f13-90c9-406e3937b715 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:28:04.160826 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-94c097e6-6cd4-4f13-90c9-406e3937b715 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:28:04.162523 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-94c097e6-6cd4-4f13-90c9-406e3937b715 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:28:04.257120 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1189/2380] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:28:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:28:04.324099 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cec5c03-2df5-4b28-b982-d7c0a03b17ea None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:28:04.324392 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cec5c03-2df5-4b28-b982-d7c0a03b17ea None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:28:04.324677 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cec5c03-2df5-4b28-b982-d7c0a03b17ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:28:04.347991 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-3cec5c03-2df5-4b28-b982-d7c0a03b17ea None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:28:04.349283 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-3cec5c03-2df5-4b28-b982-d7c0a03b17ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:28:04.360561 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1192/2381] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:28:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:28:04.364778 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c097e6-6cd4-4f13-90c9-406e3937b715 None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:28:04.379204 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c097e6-6cd4-4f13-90c9-406e3937b715 None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:28:04.384383 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c72219-df90-4b43-aa6a-259f8e4ae6b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:28:04.385162 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c72219-df90-4b43-aa6a-259f8e4ae6b2 admin admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:28:04.385510 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c72219-df90-4b43-aa6a-259f8e4ae6b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:28:04.385843 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c72219-df90-4b43-aa6a-259f8e4ae6b2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:28:04.386617 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c72219-df90-4b43-aa6a-259f8e4ae6b2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:28:04.387016 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c72219-df90-4b43-aa6a-259f8e4ae6b2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:28:04.388677 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c72219-df90-4b43-aa6a-259f8e4ae6b2 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:28:04.402909 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1190/2382] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Aug 31 01:28:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 01:28:04.410840 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cec5c03-2df5-4b28-b982-d7c0a03b17ea None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:28:04.420334 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cec5c03-2df5-4b28-b982-d7c0a03b17ea None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:28:04.425819 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc89881f-69a9-42e2-afd6-975463452b74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:28:04.425819 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc89881f-69a9-42e2-afd6-975463452b74 admin admin] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:28:04.428423 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc89881f-69a9-42e2-afd6-975463452b74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:28:04.428858 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc89881f-69a9-42e2-afd6-975463452b74 admin admin] PATH_INFO: `/v3/projects` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:28:04.429605 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc89881f-69a9-42e2-afd6-975463452b74 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:28:04.431593 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc89881f-69a9-42e2-afd6-975463452b74 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:28:04.525774 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1193/2383] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 01:28:04 2022] POST /identity/v3/projects => generated 385 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 01:28:04.529611 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c72219-df90-4b43-aa6a-259f8e4ae6b2 admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:28:04.548138 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c72219-df90-4b43-aa6a-259f8e4ae6b2 admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:28:04.564158 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:28:04.619001 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9f9f26-5a46-4ad1-ba3c-ee494768aed7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:28:04.619613 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9f9f26-5a46-4ad1-ba3c-ee494768aed7 admin admin] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:28:04.620833 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9f9f26-5a46-4ad1-ba3c-ee494768aed7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:28:04.620833 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9f9f26-5a46-4ad1-ba3c-ee494768aed7 admin admin] PATH_INFO: `/v3/users` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:28:04.620833 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9f9f26-5a46-4ad1-ba3c-ee494768aed7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:28:04.622818 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9f9f26-5a46-4ad1-ba3c-ee494768aed7 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:28:04.734520 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1191/2384] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 01:28:04 2022] POST /identity/v3/users => generated 293 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:28:04.738409 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc89881f-69a9-42e2-afd6-975463452b74 admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:28:04.749791 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc89881f-69a9-42e2-afd6-975463452b74 admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:28:04.755240 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08df1ca5-50dd-4ed7-946f-520e31df853a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:28:04.755850 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08df1ca5-50dd-4ed7-946f-520e31df853a admin admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:28:04.756134 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08df1ca5-50dd-4ed7-946f-520e31df853a admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:28:04.756411 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08df1ca5-50dd-4ed7-946f-520e31df853a admin admin] PATH_INFO: `/v3/roles` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:28:04.757091 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08df1ca5-50dd-4ed7-946f-520e31df853a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:28:04.758726 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08df1ca5-50dd-4ed7-946f-520e31df853a admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:28:04.772217 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1194/2385] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:28:04 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:28:04.780929 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9f9f26-5a46-4ad1-ba3c-ee494768aed7 admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:28:04.791355 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9f9f26-5a46-4ad1-ba3c-ee494768aed7 admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:28:04.798764 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27971e56-2855-4a04-b47f-80fc4aa10ac0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:28:04.799417 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27971e56-2855-4a04-b47f-80fc4aa10ac0 admin admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:28:04.799709 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27971e56-2855-4a04-b47f-80fc4aa10ac0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:28:04.799991 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27971e56-2855-4a04-b47f-80fc4aa10ac0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:28:04.800640 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27971e56-2855-4a04-b47f-80fc4aa10ac0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:28:04.802294 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27971e56-2855-4a04-b47f-80fc4aa10ac0 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:28:04.819696 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1192/2386] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:28:04 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 01:28:04.820911 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08df1ca5-50dd-4ed7-946f-520e31df853a admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:28:04.832092 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08df1ca5-50dd-4ed7-946f-520e31df853a admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:28:04.837895 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d35bf8a-9915-4d9f-80a5-65d18d31674a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:28:04.838775 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d35bf8a-9915-4d9f-80a5-65d18d31674a admin admin] REQUEST_METHOD: `PUT` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:28:04.839231 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d35bf8a-9915-4d9f-80a5-65d18d31674a admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:28:04.839672 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d35bf8a-9915-4d9f-80a5-65d18d31674a admin admin] PATH_INFO: `/v3/projects/9befc65a9da84a99942b803ac6a15a0b/users/cb0154f4a3704bd0b746071bdfb90420/roles/d0b9768f4d014766b9326387f2e11f34` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:28:04.856267 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d35bf8a-9915-4d9f-80a5-65d18d31674a admin admin] RBAC: Authorizing `identity:create_grant(project_id=9befc65a9da84a99942b803ac6a15a0b, user_id=cb0154f4a3704bd0b746071bdfb90420, role_id=d0b9768f4d014766b9326387f2e11f34)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:28:04.858975 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d35bf8a-9915-4d9f-80a5-65d18d31674a admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:28:04.939098 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1195/2387] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 01:28:04 2022] PUT /identity/v3/projects/9befc65a9da84a99942b803ac6a15a0b/users/cb0154f4a3704bd0b746071bdfb90420/roles/d0b9768f4d014766b9326387f2e11f34 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:28:04.944995 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d44843-abc7-4688-b526-43a3bcad4b71 None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:28:04.945290 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d44843-abc7-4688-b526-43a3bcad4b71 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:28:04.945572 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d44843-abc7-4688-b526-43a3bcad4b71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:28:04.966975 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-38d44843-abc7-4688-b526-43a3bcad4b71 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:28:04.968235 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-38d44843-abc7-4688-b526-43a3bcad4b71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:28:05.037890 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1193/2388] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:28:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:28:05.048296 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d35bf8a-9915-4d9f-80a5-65d18d31674a admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:28:05.050938 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-0d35bf8a-9915-4d9f-80a5-65d18d31674a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:28:05.116890 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d35bf8a-9915-4d9f-80a5-65d18d31674a admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:28:05.120642 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ce7822-93f4-40a2-81b4-18c00c133f29 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0817c3320d5d41d8bba0550c8ae8dd60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:28:05.121369 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ce7822-93f4-40a2-81b4-18c00c133f29 service neutron] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:28:05.121665 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ce7822-93f4-40a2-81b4-18c00c133f29 service neutron] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:28:05.121942 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ce7822-93f4-40a2-81b4-18c00c133f29 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:28:05.131901 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ce7822-93f4-40a2-81b4-18c00c133f29 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:28:05.133382 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ce7822-93f4-40a2-81b4-18c00c133f29 service neutron] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:28:05.147646 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1196/2389] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:28:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:28:12.174632 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dae264-a7a9-4f45-af8d-9f3baedf5291 None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:28:12.175766 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dae264-a7a9-4f45-af8d-9f3baedf5291 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:28:12.175889 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dae264-a7a9-4f45-af8d-9f3baedf5291 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:28:12.235095 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-a7dae264-a7a9-4f45-af8d-9f3baedf5291 None None] MFA Rules not processed for user `cb0154f4a3704bd0b746071bdfb90420`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:28:12.236288 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-a7dae264-a7a9-4f45-af8d-9f3baedf5291 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:28:12.374342 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1194/2390] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:28:12 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:28:12.378606 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ce7822-93f4-40a2-81b4-18c00c133f29 service neutron] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:28:12.389244 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ce7822-93f4-40a2-81b4-18c00c133f29 service neutron] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:28:12.398127 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d42cfb-c0b7-49ef-a878-eb16d9e47606 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:28:12.399213 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d42cfb-c0b7-49ef-a878-eb16d9e47606 admin admin] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:28:12.399509 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d42cfb-c0b7-49ef-a878-eb16d9e47606 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:28:12.399776 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d42cfb-c0b7-49ef-a878-eb16d9e47606 admin admin] PATH_INFO: `/v3/projects` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:28:12.400416 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d42cfb-c0b7-49ef-a878-eb16d9e47606 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:28:12.402405 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d42cfb-c0b7-49ef-a878-eb16d9e47606 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:28:12.739879 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1197/2391] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 01:28:12 2022] POST /identity/v3/projects => generated 385 bytes in 362 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:28:12.744249 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7dae264-a7a9-4f45-af8d-9f3baedf5291 None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:28:12.756013 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7dae264-a7a9-4f45-af8d-9f3baedf5291 None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:28:12.776395 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:28:12.827157 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46970bf6-71c3-4493-829e-90f0188706d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:28:12.827876 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46970bf6-71c3-4493-829e-90f0188706d9 admin admin] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:28:12.828232 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46970bf6-71c3-4493-829e-90f0188706d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:28:12.828582 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46970bf6-71c3-4493-829e-90f0188706d9 admin admin] PATH_INFO: `/v3/users` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:28:12.829331 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46970bf6-71c3-4493-829e-90f0188706d9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:28:12.832305 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46970bf6-71c3-4493-829e-90f0188706d9 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:28:12.933285 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1195/2392] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 01:28:12 2022] POST /identity/v3/users => generated 299 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:28:12.937437 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d42cfb-c0b7-49ef-a878-eb16d9e47606 admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:28:12.952929 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d42cfb-c0b7-49ef-a878-eb16d9e47606 admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:28:12.972769 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de20a0e3-5c96-4fd8-bfb2-c41858f16f5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:28:12.973400 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de20a0e3-5c96-4fd8-bfb2-c41858f16f5c admin admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:28:12.973685 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de20a0e3-5c96-4fd8-bfb2-c41858f16f5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:28:12.973963 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de20a0e3-5c96-4fd8-bfb2-c41858f16f5c admin admin] PATH_INFO: `/v3/roles` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:28:12.974656 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de20a0e3-5c96-4fd8-bfb2-c41858f16f5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:28:12.976304 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de20a0e3-5c96-4fd8-bfb2-c41858f16f5c admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:28:13.000977 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1198/2393] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:28:12 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:28:13.005463 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46970bf6-71c3-4493-829e-90f0188706d9 admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:28:13.020221 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46970bf6-71c3-4493-829e-90f0188706d9 admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:28:13.040687 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0d409d-6ac3-4a32-a12e-8e283bb63e30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:28:13.041484 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0d409d-6ac3-4a32-a12e-8e283bb63e30 admin admin] REQUEST_METHOD: `PUT` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:28:13.041769 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0d409d-6ac3-4a32-a12e-8e283bb63e30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:28:13.042050 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0d409d-6ac3-4a32-a12e-8e283bb63e30 admin admin] PATH_INFO: `/v3/projects/5987e5c9567141e09f55b0e723d2133d/users/340bc7c2d6d44ccb84e04909e0ff1514/roles/d009411496b8452b88e77ffb8db015a7` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:28:13.068366 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0d409d-6ac3-4a32-a12e-8e283bb63e30 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5987e5c9567141e09f55b0e723d2133d, user_id=340bc7c2d6d44ccb84e04909e0ff1514, role_id=d009411496b8452b88e77ffb8db015a7)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:28:13.071311 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0d409d-6ac3-4a32-a12e-8e283bb63e30 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:28:13.142165 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1196/2394] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 01:28:13 2022] PUT /identity/v3/projects/5987e5c9567141e09f55b0e723d2133d/users/340bc7c2d6d44ccb84e04909e0ff1514/roles/d009411496b8452b88e77ffb8db015a7 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:28:13.146428 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de20a0e3-5c96-4fd8-bfb2-c41858f16f5c admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:28:13.164921 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de20a0e3-5c96-4fd8-bfb2-c41858f16f5c admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:28:13.248066 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:28:13.272902 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5d11f9-5533-4d52-a2f2-3dd3236e1871 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:28:13.274290 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5d11f9-5533-4d52-a2f2-3dd3236e1871 admin admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:28:13.274926 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5d11f9-5533-4d52-a2f2-3dd3236e1871 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:28:13.275510 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5d11f9-5533-4d52-a2f2-3dd3236e1871 admin admin] PATH_INFO: `/v3/roles` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:28:13.276328 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5d11f9-5533-4d52-a2f2-3dd3236e1871 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:28:13.278270 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5d11f9-5533-4d52-a2f2-3dd3236e1871 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:28:13.303325 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0d409d-6ac3-4a32-a12e-8e283bb63e30 admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:28:13.327682 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1199/2395] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:28:13 2022] GET /identity/v3/roles => generated 1420 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:28:13.337912 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0d409d-6ac3-4a32-a12e-8e283bb63e30 admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:28:13.350257 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca00a36-2cdf-405d-9e91-b1302199ac27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:28:13.350257 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca00a36-2cdf-405d-9e91-b1302199ac27 admin admin] REQUEST_METHOD: `PUT` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:28:13.351316 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca00a36-2cdf-405d-9e91-b1302199ac27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:28:13.351316 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca00a36-2cdf-405d-9e91-b1302199ac27 admin admin] PATH_INFO: `/v3/projects/5987e5c9567141e09f55b0e723d2133d/users/340bc7c2d6d44ccb84e04909e0ff1514/roles/d009411496b8452b88e77ffb8db015a7` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:28:13.361364 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca00a36-2cdf-405d-9e91-b1302199ac27 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5987e5c9567141e09f55b0e723d2133d, user_id=340bc7c2d6d44ccb84e04909e0ff1514, role_id=d009411496b8452b88e77ffb8db015a7)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:28:13.370370 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca00a36-2cdf-405d-9e91-b1302199ac27 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:28:13.400865 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1197/2396] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 01:28:13 2022] PUT /identity/v3/projects/5987e5c9567141e09f55b0e723d2133d/users/340bc7c2d6d44ccb84e04909e0ff1514/roles/d009411496b8452b88e77ffb8db015a7 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 01:28:18.663879 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5d11f9-5533-4d52-a2f2-3dd3236e1871 admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:28:18.665815 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-0f5d11f9-5533-4d52-a2f2-3dd3236e1871 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:28:18.756462 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5d11f9-5533-4d52-a2f2-3dd3236e1871 admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:28:18.872510 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b35da0-5a9d-4566-a4b6-72b7331733ba service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8744c9de803048f583d0317d01f6a540', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:28:18.873307 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b35da0-5a9d-4566-a4b6-72b7331733ba service nova] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:28:18.873553 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b35da0-5a9d-4566-a4b6-72b7331733ba service nova] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:28:18.873860 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b35da0-5a9d-4566-a4b6-72b7331733ba service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:28:18.880805 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-b6b35da0-5a9d-4566-a4b6-72b7331733ba service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:28:18.907497 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b35da0-5a9d-4566-a4b6-72b7331733ba service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:28:18.909752 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b35da0-5a9d-4566-a4b6-72b7331733ba service nova] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:28:18.935007 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1200/2397] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:28:18 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 271 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:28:19.223403 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1391e00a-222c-4424-97e8-6869980e849e None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:28:19.224210 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1391e00a-222c-4424-97e8-6869980e849e None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:28:19.224504 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1391e00a-222c-4424-97e8-6869980e849e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:28:19.273570 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-1391e00a-222c-4424-97e8-6869980e849e None None] MFA Rules not processed for user `340bc7c2d6d44ccb84e04909e0ff1514`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:28:19.277102 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-1391e00a-222c-4424-97e8-6869980e849e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:28:19.380068 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1198/2398] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:28:19 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:28:19.394625 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b35da0-5a9d-4566-a4b6-72b7331733ba service nova] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:28:19.399410 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-b6b35da0-5a9d-4566-a4b6-72b7331733ba service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:28:19.457063 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b35da0-5a9d-4566-a4b6-72b7331733ba service nova] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:28:19.460756 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051eaf36-b773-4c63-bc81-c32f1429658e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8590c288028418ebe7a4163fd8039f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:28:19.467298 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051eaf36-b773-4c63-bc81-c32f1429658e service glance] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:28:19.467298 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051eaf36-b773-4c63-bc81-c32f1429658e service glance] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:28:19.467298 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051eaf36-b773-4c63-bc81-c32f1429658e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:28:19.483845 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:28:19.526463 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051eaf36-b773-4c63-bc81-c32f1429658e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:28:19.528576 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051eaf36-b773-4c63-bc81-c32f1429658e service glance] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:28:19.538920 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1201/2399] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:28:19 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:28:19.819311 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129600db-d683-465f-8d1a-e89e8944246c None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:28:19.819602 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129600db-d683-465f-8d1a-e89e8944246c None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:28:19.819880 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129600db-d683-465f-8d1a-e89e8944246c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:28:19.842321 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-129600db-d683-465f-8d1a-e89e8944246c None None] MFA Rules not processed for user `b3efb4f9daac4a0ca05a36dd361f17e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:28:19.843646 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-129600db-d683-465f-8d1a-e89e8944246c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:28:19.902856 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1199/2400] 10.222.0.20 () {58 vars in 1009 bytes} [Wed Aug 31 01:28:19 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:28:19.917043 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051eaf36-b773-4c63-bc81-c32f1429658e service glance] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:28:19.918967 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-051eaf36-b773-4c63-bc81-c32f1429658e service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:28:20.010259 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051eaf36-b773-4c63-bc81-c32f1429658e service glance] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:28:20.016940 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f78c41-5a65-46b3-8e47-c81a59410106 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc1d010919ec40878ae829cc8dceec4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:28:20.017562 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f78c41-5a65-46b3-8e47-c81a59410106 service swift] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:28:20.017861 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f78c41-5a65-46b3-8e47-c81a59410106 service swift] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:28:20.018171 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f78c41-5a65-46b3-8e47-c81a59410106 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:28:20.027265 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f78c41-5a65-46b3-8e47-c81a59410106 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:28:20.028940 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f78c41-5a65-46b3-8e47-c81a59410106 service swift] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:28:20.038265 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1202/2401] 10.222.0.20 () {60 vars in 1384 bytes} [Wed Aug 31 01:28:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 01:28:21.267346 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d178f6-726b-4de5-89de-f6f59e48204e None None] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:28:21.268307 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d178f6-726b-4de5-89de-f6f59e48204e None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:28:21.269246 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d178f6-726b-4de5-89de-f6f59e48204e None None] PATH_INFO: `/` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:28:21.271616 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1200/2402] 10.222.0.20 () {54 vars in 921 bytes} [Wed Aug 31 01:28:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:28:21.293472 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ede10d-61de-4ef0-ab20-811db10da87c None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:28:21.294069 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ede10d-61de-4ef0-ab20-811db10da87c None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:28:21.294442 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ede10d-61de-4ef0-ab20-811db10da87c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:28:21.350023 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-37ede10d-61de-4ef0-ab20-811db10da87c None None] MFA Rules not processed for user `8744c9de803048f583d0317d01f6a540`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:28:21.351504 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-37ede10d-61de-4ef0-ab20-811db10da87c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:28:21.372702 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1203/2403] 10.222.0.20 () {58 vars in 1018 bytes} [Wed Aug 31 01:28:21 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:28:21.385510 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d178f6-726b-4de5-89de-f6f59e48204e None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:28:21.401833 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d178f6-726b-4de5-89de-f6f59e48204e None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:28:21.405857 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65613e4a-517a-4c49-954e-ebe6510f7569 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8590c288028418ebe7a4163fd8039f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:28:21.406596 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65613e4a-517a-4c49-954e-ebe6510f7569 service glance] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:28:21.406970 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65613e4a-517a-4c49-954e-ebe6510f7569 service glance] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:28:21.407425 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65613e4a-517a-4c49-954e-ebe6510f7569 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:28:21.419978 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:28:21.424297 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65613e4a-517a-4c49-954e-ebe6510f7569 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:28:21.427044 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65613e4a-517a-4c49-954e-ebe6510f7569 service glance] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:28:21.442488 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1201/2404] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:28:21 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:28:21.654762 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed60d993-3b72-4490-9354-68c85366d702 None None] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:28:21.655491 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed60d993-3b72-4490-9354-68c85366d702 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:28:21.656005 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed60d993-3b72-4490-9354-68c85366d702 None None] PATH_INFO: `/` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:28:21.657830 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1204/2405] 10.222.0.20 () {54 vars in 921 bytes} [Wed Aug 31 01:28:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:28:21.663248 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7881b41-2816-46d0-8de3-9b9831b0259b None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:28:21.663543 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7881b41-2816-46d0-8de3-9b9831b0259b None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:28:21.663849 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7881b41-2816-46d0-8de3-9b9831b0259b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:28:21.695455 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-c7881b41-2816-46d0-8de3-9b9831b0259b None None] MFA Rules not processed for user `0817c3320d5d41d8bba0550c8ae8dd60`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:28:21.697507 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-c7881b41-2816-46d0-8de3-9b9831b0259b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:28:21.757722 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1202/2406] 10.222.0.20 () {58 vars in 1018 bytes} [Wed Aug 31 01:28:21 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:28:21.770855 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed60d993-3b72-4490-9354-68c85366d702 None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:28:21.774080 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-ed60d993-3b72-4490-9354-68c85366d702 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:28:21.788676 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed60d993-3b72-4490-9354-68c85366d702 None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:28:21.793245 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65803108-2ab2-42f6-9f51-21d7239613b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0817c3320d5d41d8bba0550c8ae8dd60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:28:21.794169 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65803108-2ab2-42f6-9f51-21d7239613b3 service neutron] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:28:21.794516 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65803108-2ab2-42f6-9f51-21d7239613b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:28:21.794802 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65803108-2ab2-42f6-9f51-21d7239613b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:28:21.803889 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65803108-2ab2-42f6-9f51-21d7239613b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:28:21.805384 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65803108-2ab2-42f6-9f51-21d7239613b3 service neutron] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:28:21.817264 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1205/2407] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:28:21 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:28:21.919036 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24be1287-9c76-4865-b361-9fdf4078b73f None None] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:28:21.919594 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24be1287-9c76-4865-b361-9fdf4078b73f None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:28:21.920054 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24be1287-9c76-4865-b361-9fdf4078b73f None None] PATH_INFO: `/` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:28:21.922076 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1203/2408] 10.222.0.20 () {54 vars in 921 bytes} [Wed Aug 31 01:28:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 01:28:21.928075 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa82331f-ceae-4396-8616-69ced573f0fb None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:28:21.928365 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa82331f-ceae-4396-8616-69ced573f0fb None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:28:21.928650 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa82331f-ceae-4396-8616-69ced573f0fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:28:21.953215 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-fa82331f-ceae-4396-8616-69ced573f0fb None None] MFA Rules not processed for user `8744c9de803048f583d0317d01f6a540`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:28:21.954717 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-fa82331f-ceae-4396-8616-69ced573f0fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:28:21.964715 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1206/2409] 10.222.0.20 () {58 vars in 1018 bytes} [Wed Aug 31 01:28:21 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:28:21.979891 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24be1287-9c76-4865-b361-9fdf4078b73f None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:28:21.998554 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24be1287-9c76-4865-b361-9fdf4078b73f None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:28:22.005353 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f80068-cfba-49b9-bf85-d774731efed0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0817c3320d5d41d8bba0550c8ae8dd60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:28:22.006024 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f80068-cfba-49b9-bf85-d774731efed0 service neutron] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:28:22.006307 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f80068-cfba-49b9-bf85-d774731efed0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:28:22.006586 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f80068-cfba-49b9-bf85-d774731efed0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:28:22.018520 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f80068-cfba-49b9-bf85-d774731efed0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:28:22.019981 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f80068-cfba-49b9-bf85-d774731efed0 service neutron] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:28:22.032567 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1204/2410] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:28:21 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:28:25.160642 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed77b7e5-f3f3-4617-ba86-0180e5692a77 None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:28:25.161171 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed77b7e5-f3f3-4617-ba86-0180e5692a77 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:28:25.161268 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed77b7e5-f3f3-4617-ba86-0180e5692a77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:28:25.182226 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-ed77b7e5-f3f3-4617-ba86-0180e5692a77 None None] MFA Rules not processed for user `b3efb4f9daac4a0ca05a36dd361f17e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:28:25.184415 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-ed77b7e5-f3f3-4617-ba86-0180e5692a77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:28:25.194557 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1207/2411] 10.222.0.20 () {58 vars in 1009 bytes} [Wed Aug 31 01:28:25 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:28:25.215519 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f80068-cfba-49b9-bf85-d774731efed0 service neutron] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:28:25.244289 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f80068-cfba-49b9-bf85-d774731efed0 service neutron] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:28:25.249570 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0082d8ba-c91d-49b4-92ff-9df6117cf73a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc1d010919ec40878ae829cc8dceec4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:28:25.250190 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0082d8ba-c91d-49b4-92ff-9df6117cf73a service swift] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:28:25.250476 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0082d8ba-c91d-49b4-92ff-9df6117cf73a service swift] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:28:25.250890 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0082d8ba-c91d-49b4-92ff-9df6117cf73a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:28:25.260630 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0082d8ba-c91d-49b4-92ff-9df6117cf73a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:28:25.262616 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0082d8ba-c91d-49b4-92ff-9df6117cf73a service swift] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:28:25.289890 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1205/2412] 10.222.0.20 () {60 vars in 1384 bytes} [Wed Aug 31 01:28:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 01:28:32.654302 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a84a83-839a-446a-9425-74c6bbce7fed None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:28:32.654869 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a84a83-839a-446a-9425-74c6bbce7fed None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:28:32.655167 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a84a83-839a-446a-9425-74c6bbce7fed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:28:32.676136 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-88a84a83-839a-446a-9425-74c6bbce7fed None None] MFA Rules not processed for user `cb0154f4a3704bd0b746071bdfb90420`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:28:32.677786 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-88a84a83-839a-446a-9425-74c6bbce7fed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:28:32.685574 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1208/2413] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:28:32 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:28:35.273753 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=38 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:28:38.580449 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a562aea5-915f-47e4-a130-5262e401b955 None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:28:38.581103 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a562aea5-915f-47e4-a130-5262e401b955 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:28:38.581209 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a562aea5-915f-47e4-a130-5262e401b955 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:28:38.603927 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-a562aea5-915f-47e4-a130-5262e401b955 None None] MFA Rules not processed for user `b3efb4f9daac4a0ca05a36dd361f17e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:28:38.605218 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-a562aea5-915f-47e4-a130-5262e401b955 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:28:38.617041 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1206/2414] 10.222.0.20 () {58 vars in 1009 bytes} [Wed Aug 31 01:28:38 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:28:38.634983 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a84a83-839a-446a-9425-74c6bbce7fed None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:28:38.650497 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a84a83-839a-446a-9425-74c6bbce7fed None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:28:38.656922 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50498b63-f812-41e2-af3a-71069b8ca425 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc1d010919ec40878ae829cc8dceec4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:28:38.657944 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50498b63-f812-41e2-af3a-71069b8ca425 service swift] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:28:38.658549 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50498b63-f812-41e2-af3a-71069b8ca425 service swift] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:28:38.659018 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50498b63-f812-41e2-af3a-71069b8ca425 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:28:38.671236 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50498b63-f812-41e2-af3a-71069b8ca425 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:28:38.673229 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50498b63-f812-41e2-af3a-71069b8ca425 service swift] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:28:38.683804 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1209/2415] 10.222.0.20 () {60 vars in 1384 bytes} [Wed Aug 31 01:28:38 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 01:28:47.043981 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a562aea5-915f-47e4-a130-5262e401b955 None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:28:47.045404 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-a562aea5-915f-47e4-a130-5262e401b955 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:28:47.104221 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a562aea5-915f-47e4-a130-5262e401b955 None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:28:47.107790 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e04674f-d946-4290-ab99-52c7c6c2053b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5c8fc69c518412ba1aad61a9f4f558c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:28:47.108402 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e04674f-d946-4290-ab99-52c7c6c2053b service cinder] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:28:47.108686 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e04674f-d946-4290-ab99-52c7c6c2053b service cinder] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:28:47.108989 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e04674f-d946-4290-ab99-52c7c6c2053b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:28:47.110827 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-3e04674f-d946-4290-ab99-52c7c6c2053b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:28:47.170699 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e04674f-d946-4290-ab99-52c7c6c2053b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:28:47.172184 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e04674f-d946-4290-ab99-52c7c6c2053b service cinder] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:28:47.182104 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1207/2416] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:28:47 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:28:48.463820 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50498b63-f812-41e2-af3a-71069b8ca425 service swift] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:28:48.478920 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50498b63-f812-41e2-af3a-71069b8ca425 service swift] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:28:48.492986 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c1c8cb-2496-47d1-acb8-3f49597bd2bf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8744c9de803048f583d0317d01f6a540', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:28:48.493926 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c1c8cb-2496-47d1-acb8-3f49597bd2bf service nova] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:28:48.494803 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c1c8cb-2496-47d1-acb8-3f49597bd2bf service nova] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:28:48.495252 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c1c8cb-2496-47d1-acb8-3f49597bd2bf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:28:48.499261 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-95c1c8cb-2496-47d1-acb8-3f49597bd2bf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:28:48.519899 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c1c8cb-2496-47d1-acb8-3f49597bd2bf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:28:48.519899 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c1c8cb-2496-47d1-acb8-3f49597bd2bf service nova] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:28:48.533807 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1210/2417] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:28:48 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:28:50.613977 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc05838c-ce66-48cb-9aad-0df180a9f848 None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:28:50.614511 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc05838c-ce66-48cb-9aad-0df180a9f848 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:28:50.614511 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc05838c-ce66-48cb-9aad-0df180a9f848 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:28:50.634708 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-fc05838c-ce66-48cb-9aad-0df180a9f848 None None] MFA Rules not processed for user `b3efb4f9daac4a0ca05a36dd361f17e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:28:50.635935 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-fc05838c-ce66-48cb-9aad-0df180a9f848 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:28:50.643128 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1208/2418] 10.222.0.20 () {58 vars in 967 bytes} [Wed Aug 31 01:28:50 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:28:50.655307 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c1c8cb-2496-47d1-acb8-3f49597bd2bf service nova] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:28:50.661125 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=81 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:28:50.667260 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c1c8cb-2496-47d1-acb8-3f49597bd2bf service nova] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:28:50.672663 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac911b8f-ebed-46fb-b0e2-8f77125d0563 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc1d010919ec40878ae829cc8dceec4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:28:50.677053 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac911b8f-ebed-46fb-b0e2-8f77125d0563 service swift] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:28:50.677053 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac911b8f-ebed-46fb-b0e2-8f77125d0563 service swift] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:28:50.677053 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac911b8f-ebed-46fb-b0e2-8f77125d0563 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:28:50.682416 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac911b8f-ebed-46fb-b0e2-8f77125d0563 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:28:50.684009 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac911b8f-ebed-46fb-b0e2-8f77125d0563 service swift] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:28:50.692623 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1211/2419] 10.222.0.20 () {60 vars in 1384 bytes} [Wed Aug 31 01:28:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 01:29:00.628242 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=54 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:29:00.697134 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=5 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:29:02.032604 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc05838c-ce66-48cb-9aad-0df180a9f848 None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:29:02.042404 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc05838c-ce66-48cb-9aad-0df180a9f848 None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:29:02.046029 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f92747-345f-4059-96f3-85524b641cac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0817c3320d5d41d8bba0550c8ae8dd60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:29:02.046627 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f92747-345f-4059-96f3-85524b641cac service neutron] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:29:02.046904 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f92747-345f-4059-96f3-85524b641cac service neutron] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:29:02.047175 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f92747-345f-4059-96f3-85524b641cac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:29:02.048946 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-49f92747-345f-4059-96f3-85524b641cac service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:29:02.113660 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f92747-345f-4059-96f3-85524b641cac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:29:02.115588 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f92747-345f-4059-96f3-85524b641cac service neutron] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:29:02.125257 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1209/2420] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:29:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:29:05.683076 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac911b8f-ebed-46fb-b0e2-8f77125d0563 service swift] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:29:05.684538 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-ac911b8f-ebed-46fb-b0e2-8f77125d0563 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:29:05.756819 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac911b8f-ebed-46fb-b0e2-8f77125d0563 service swift] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:29:05.762042 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868f6808-2762-4e3d-9aec-611fc2e45f21 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dea0fe5b1d6b405cb47b95192580303b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:29:05.762675 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868f6808-2762-4e3d-9aec-611fc2e45f21 service placement] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:29:05.762944 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868f6808-2762-4e3d-9aec-611fc2e45f21 service placement] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:29:05.763207 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868f6808-2762-4e3d-9aec-611fc2e45f21 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:29:05.764952 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-868f6808-2762-4e3d-9aec-611fc2e45f21 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:29:05.776938 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868f6808-2762-4e3d-9aec-611fc2e45f21 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:29:05.778522 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868f6808-2762-4e3d-9aec-611fc2e45f21 service placement] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:29:05.788160 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1212/2421] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:29:05 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:29:05.846306 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f92747-345f-4059-96f3-85524b641cac service neutron] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:29:05.847760 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-49f92747-345f-4059-96f3-85524b641cac service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:29:05.859759 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f92747-345f-4059-96f3-85524b641cac service neutron] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:29:05.864507 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2153bf-ccb8-4a67-b331-8730b1b3f9a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:29:05.865107 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2153bf-ccb8-4a67-b331-8730b1b3f9a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:29:05.865393 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2153bf-ccb8-4a67-b331-8730b1b3f9a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:29:05.865673 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2153bf-ccb8-4a67-b331-8730b1b3f9a6 admin admin] PATH_INFO: `/v3/users/52ed99737fd240789bb682fd9bf11be3` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:29:05.866867 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2153bf-ccb8-4a67-b331-8730b1b3f9a6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=52ed99737fd240789bb682fd9bf11be3)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:29:05.868846 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2153bf-ccb8-4a67-b331-8730b1b3f9a6 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:29:06.042225 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-be2153bf-ccb8-4a67-b331-8730b1b3f9a6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '52ed99737fd240789bb682fd9bf11be3'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:29:06.148434 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-be2153bf-ccb8-4a67-b331-8730b1b3f9a6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '52ed99737fd240789bb682fd9bf11be3'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:29:06.149025 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-be2153bf-ccb8-4a67-b331-8730b1b3f9a6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '52ed99737fd240789bb682fd9bf11be3'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:29:06.156143 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-be2153bf-ccb8-4a67-b331-8730b1b3f9a6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '52ed99737fd240789bb682fd9bf11be3'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:29:06.156788 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-be2153bf-ccb8-4a67-b331-8730b1b3f9a6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '52ed99737fd240789bb682fd9bf11be3'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:29:06.173675 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1210/2422] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 01:29:05 2022] DELETE /identity/v3/users/52ed99737fd240789bb682fd9bf11be3 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:29:06.435863 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868f6808-2762-4e3d-9aec-611fc2e45f21 service placement] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:29:06.447201 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868f6808-2762-4e3d-9aec-611fc2e45f21 service placement] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:29:06.504477 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8595fa21-0b60-464e-9410-914681d52032 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:29:06.505169 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8595fa21-0b60-464e-9410-914681d52032 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:29:06.505457 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8595fa21-0b60-464e-9410-914681d52032 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:29:06.505735 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8595fa21-0b60-464e-9410-914681d52032 admin admin] PATH_INFO: `/v3/projects/164da7d4776d4e9f8cf6497aa784d312` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:29:06.506864 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8595fa21-0b60-464e-9410-914681d52032 admin admin] RBAC: Authorizing `identity:delete_project(project_id=164da7d4776d4e9f8cf6497aa784d312)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:29:06.508787 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8595fa21-0b60-464e-9410-914681d52032 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:29:06.622872 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-8595fa21-0b60-464e-9410-914681d52032 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '164da7d4776d4e9f8cf6497aa784d312', 'request_id': 'req-8595fa21-0b60-464e-9410-914681d52032'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:29:06.628339 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-8595fa21-0b60-464e-9410-914681d52032 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '164da7d4776d4e9f8cf6497aa784d312', 'request_id': 'req-8595fa21-0b60-464e-9410-914681d52032'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:29:06.702349 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-8595fa21-0b60-464e-9410-914681d52032 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '164da7d4776d4e9f8cf6497aa784d312', 'request_id': 'req-8595fa21-0b60-464e-9410-914681d52032'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:29:06.714283 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-8595fa21-0b60-464e-9410-914681d52032 admin admin] The token cache is being invalidate because project 164da7d4776d4e9f8cf6497aa784d312 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81750) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 01:29:06.714576 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-8595fa21-0b60-464e-9410-914681d52032 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:29:06.715213 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-8595fa21-0b60-464e-9410-914681d52032 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:29:06.716735 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1213/2423] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 01:29:06 2022] DELETE /identity/v3/projects/164da7d4776d4e9f8cf6497aa784d312 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:29:06.721648 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4971ba-0d40-4405-b2d8-b7b1c711b7fd None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:29:06.721944 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4971ba-0d40-4405-b2d8-b7b1c711b7fd None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:29:06.722229 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4971ba-0d40-4405-b2d8-b7b1c711b7fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:29:06.742444 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-1a4971ba-0d40-4405-b2d8-b7b1c711b7fd None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:29:06.743636 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-1a4971ba-0d40-4405-b2d8-b7b1c711b7fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:29:06.801211 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1211/2424] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:29:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:29:06.808884 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333fc759-c47f-4e2b-afc6-30deb5153e8e None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:29:06.809180 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333fc759-c47f-4e2b-afc6-30deb5153e8e None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:29:06.809460 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333fc759-c47f-4e2b-afc6-30deb5153e8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:29:06.829069 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-333fc759-c47f-4e2b-afc6-30deb5153e8e None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:29:06.830551 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-333fc759-c47f-4e2b-afc6-30deb5153e8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:29:06.839076 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1214/2425] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:29:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:29:06.843168 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4971ba-0d40-4405-b2d8-b7b1c711b7fd None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:29:06.853535 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4971ba-0d40-4405-b2d8-b7b1c711b7fd None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:29:06.858514 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95c8f84-96c5-4881-a50b-3bffc59dee6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:29:06.859374 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95c8f84-96c5-4881-a50b-3bffc59dee6a admin admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:29:06.859889 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95c8f84-96c5-4881-a50b-3bffc59dee6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:29:06.860394 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95c8f84-96c5-4881-a50b-3bffc59dee6a admin admin] PATH_INFO: `/v3/domains` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:29:06.861160 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95c8f84-96c5-4881-a50b-3bffc59dee6a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:29:06.861473 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95c8f84-96c5-4881-a50b-3bffc59dee6a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:29:06.863125 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95c8f84-96c5-4881-a50b-3bffc59dee6a admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:29:06.868493 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:29:06.880751 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1212/2426] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Aug 31 01:29:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:29:06.908429 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333fc759-c47f-4e2b-afc6-30deb5153e8e None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:29:06.919316 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333fc759-c47f-4e2b-afc6-30deb5153e8e None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:29:06.925508 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57bf9d6-7e9b-4bc6-9a20-1bb2018e9453 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:29:06.926364 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57bf9d6-7e9b-4bc6-9a20-1bb2018e9453 admin admin] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:29:06.926668 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57bf9d6-7e9b-4bc6-9a20-1bb2018e9453 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:29:06.927039 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57bf9d6-7e9b-4bc6-9a20-1bb2018e9453 admin admin] PATH_INFO: `/v3/projects` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:29:06.927722 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57bf9d6-7e9b-4bc6-9a20-1bb2018e9453 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:29:06.929966 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57bf9d6-7e9b-4bc6-9a20-1bb2018e9453 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:29:06.941674 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:29:07.013239 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1215/2427] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 01:29:06 2022] POST /identity/v3/projects => generated 373 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:29:07.017541 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95c8f84-96c5-4881-a50b-3bffc59dee6a admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:29:07.028083 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95c8f84-96c5-4881-a50b-3bffc59dee6a admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:29:07.085895 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6955917-ec59-42c3-9eb8-1669b66708ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:29:07.086503 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6955917-ec59-42c3-9eb8-1669b66708ef admin admin] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:29:07.086798 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6955917-ec59-42c3-9eb8-1669b66708ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:29:07.087080 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6955917-ec59-42c3-9eb8-1669b66708ef admin admin] PATH_INFO: `/v3/users` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:29:07.087705 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6955917-ec59-42c3-9eb8-1669b66708ef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:29:07.089695 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6955917-ec59-42c3-9eb8-1669b66708ef admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:29:07.187963 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1213/2428] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 01:29:07 2022] POST /identity/v3/users => generated 287 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:29:07.192021 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57bf9d6-7e9b-4bc6-9a20-1bb2018e9453 admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:29:07.203016 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57bf9d6-7e9b-4bc6-9a20-1bb2018e9453 admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:29:07.207894 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56394586-6d6a-4595-9e18-3e87bd226c6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:29:07.208544 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56394586-6d6a-4595-9e18-3e87bd226c6c admin admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:29:07.209425 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56394586-6d6a-4595-9e18-3e87bd226c6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:29:07.209734 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56394586-6d6a-4595-9e18-3e87bd226c6c admin admin] PATH_INFO: `/v3/roles` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:29:07.210398 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56394586-6d6a-4595-9e18-3e87bd226c6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:29:07.212008 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56394586-6d6a-4595-9e18-3e87bd226c6c admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:29:07.222454 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1216/2429] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:29:07 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:29:07.226332 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6955917-ec59-42c3-9eb8-1669b66708ef admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:29:07.236238 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6955917-ec59-42c3-9eb8-1669b66708ef admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:29:07.240955 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b059c950-c99f-412e-bf93-e6dc59041af4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:29:07.241545 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b059c950-c99f-412e-bf93-e6dc59041af4 admin admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:29:07.241831 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b059c950-c99f-412e-bf93-e6dc59041af4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:29:07.242113 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b059c950-c99f-412e-bf93-e6dc59041af4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:29:07.242746 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b059c950-c99f-412e-bf93-e6dc59041af4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:29:07.244376 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b059c950-c99f-412e-bf93-e6dc59041af4 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:29:07.255450 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1214/2430] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:29:07 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:29:07.259483 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56394586-6d6a-4595-9e18-3e87bd226c6c admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:29:07.272543 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56394586-6d6a-4595-9e18-3e87bd226c6c admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:29:07.277405 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f1e44e-d18c-41d8-a82d-7149c99da2e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:29:07.278060 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f1e44e-d18c-41d8-a82d-7149c99da2e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:29:07.278341 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f1e44e-d18c-41d8-a82d-7149c99da2e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:29:07.278666 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f1e44e-d18c-41d8-a82d-7149c99da2e3 admin admin] PATH_INFO: `/v3/projects/ed11221969d14b2bb6ba403dd9c412a1/users/66b52863c45b42dda4dad08e3b809ad0/roles/d0b9768f4d014766b9326387f2e11f34` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:29:07.293301 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f1e44e-d18c-41d8-a82d-7149c99da2e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed11221969d14b2bb6ba403dd9c412a1, user_id=66b52863c45b42dda4dad08e3b809ad0, role_id=d0b9768f4d014766b9326387f2e11f34)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:29:07.296041 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f1e44e-d18c-41d8-a82d-7149c99da2e3 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:29:07.411555 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1217/2431] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 01:29:07 2022] PUT /identity/v3/projects/ed11221969d14b2bb6ba403dd9c412a1/users/66b52863c45b42dda4dad08e3b809ad0/roles/d0b9768f4d014766b9326387f2e11f34 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:29:07.422501 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bd1d40-b565-4258-8083-80098e8a2fdb None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:29:07.422792 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bd1d40-b565-4258-8083-80098e8a2fdb None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:29:07.423115 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bd1d40-b565-4258-8083-80098e8a2fdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:29:07.451136 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-e9bd1d40-b565-4258-8083-80098e8a2fdb None None] MFA Rules not processed for user `66b52863c45b42dda4dad08e3b809ad0`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:29:07.452346 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-e9bd1d40-b565-4258-8083-80098e8a2fdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:29:07.516645 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1215/2432] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:29:07 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 01:29:07.529050 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f1e44e-d18c-41d8-a82d-7149c99da2e3 admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:29:07.530768 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-32f1e44e-d18c-41d8-a82d-7149c99da2e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:29:07.602492 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f1e44e-d18c-41d8-a82d-7149c99da2e3 admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:29:07.608106 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d652883-ae2e-44fe-b76b-61ab44943260 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8744c9de803048f583d0317d01f6a540', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:29:07.608714 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d652883-ae2e-44fe-b76b-61ab44943260 service nova] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:29:07.609032 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d652883-ae2e-44fe-b76b-61ab44943260 service nova] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:29:07.609306 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d652883-ae2e-44fe-b76b-61ab44943260 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:29:07.617642 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d652883-ae2e-44fe-b76b-61ab44943260 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:29:07.619213 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d652883-ae2e-44fe-b76b-61ab44943260 service nova] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:29:07.628263 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1218/2433] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:29:07 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:29:07.832455 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bd1d40-b565-4258-8083-80098e8a2fdb None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:29:07.895831 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bd1d40-b565-4258-8083-80098e8a2fdb None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:29:07.902848 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d948df57-5233-49af-b4c8-3dad3687b8ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:29:07.903498 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d948df57-5233-49af-b4c8-3dad3687b8ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:29:07.903787 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d948df57-5233-49af-b4c8-3dad3687b8ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:29:07.904060 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d948df57-5233-49af-b4c8-3dad3687b8ff admin admin] PATH_INFO: `/v3/users/66b52863c45b42dda4dad08e3b809ad0` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:29:07.905219 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d948df57-5233-49af-b4c8-3dad3687b8ff admin admin] RBAC: Authorizing `identity:delete_user(user_id=66b52863c45b42dda4dad08e3b809ad0)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:29:07.907203 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d948df57-5233-49af-b4c8-3dad3687b8ff admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:29:07.909910 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:29:08.097247 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-d948df57-5233-49af-b4c8-3dad3687b8ff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '66b52863c45b42dda4dad08e3b809ad0'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:29:08.182237 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-d948df57-5233-49af-b4c8-3dad3687b8ff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '66b52863c45b42dda4dad08e3b809ad0'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:29:08.182888 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-d948df57-5233-49af-b4c8-3dad3687b8ff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '66b52863c45b42dda4dad08e3b809ad0'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:29:08.190488 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-d948df57-5233-49af-b4c8-3dad3687b8ff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '66b52863c45b42dda4dad08e3b809ad0'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:29:08.191136 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-d948df57-5233-49af-b4c8-3dad3687b8ff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '66b52863c45b42dda4dad08e3b809ad0'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:29:08.208988 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1216/2434] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 01:29:07 2022] DELETE /identity/v3/users/66b52863c45b42dda4dad08e3b809ad0 => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:29:08.214068 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6286f09a-771a-40d5-8fcd-7a29a0e1ca92 None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:29:08.214544 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6286f09a-771a-40d5-8fcd-7a29a0e1ca92 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:29:08.214673 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6286f09a-771a-40d5-8fcd-7a29a0e1ca92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:29:08.234137 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-6286f09a-771a-40d5-8fcd-7a29a0e1ca92 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:29:08.235327 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-6286f09a-771a-40d5-8fcd-7a29a0e1ca92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:29:08.293266 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1219/2435] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:29:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:29:08.303042 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d948df57-5233-49af-b4c8-3dad3687b8ff admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:29:08.304478 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-d948df57-5233-49af-b4c8-3dad3687b8ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:29:08.371963 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d948df57-5233-49af-b4c8-3dad3687b8ff admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:29:08.375675 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862f77ff-9467-423c-89b9-1a4f8a8640e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0817c3320d5d41d8bba0550c8ae8dd60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:29:08.376292 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862f77ff-9467-423c-89b9-1a4f8a8640e9 service neutron] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:29:08.376576 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862f77ff-9467-423c-89b9-1a4f8a8640e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:29:08.376891 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862f77ff-9467-423c-89b9-1a4f8a8640e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:29:08.397991 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862f77ff-9467-423c-89b9-1a4f8a8640e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:29:08.404776 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862f77ff-9467-423c-89b9-1a4f8a8640e9 service neutron] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:29:08.412055 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1217/2436] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:29:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:29:08.734343 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6286f09a-771a-40d5-8fcd-7a29a0e1ca92 None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:29:08.744012 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6286f09a-771a-40d5-8fcd-7a29a0e1ca92 None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:29:08.753119 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e342b8e7-a870-4eeb-a2b7-070ec217f800 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0817c3320d5d41d8bba0550c8ae8dd60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:29:08.753746 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e342b8e7-a870-4eeb-a2b7-070ec217f800 service neutron] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:29:08.754032 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e342b8e7-a870-4eeb-a2b7-070ec217f800 service neutron] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:29:08.754309 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e342b8e7-a870-4eeb-a2b7-070ec217f800 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:29:08.756627 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-e342b8e7-a870-4eeb-a2b7-070ec217f800 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:29:08.799013 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:29:08.814098 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862f77ff-9467-423c-89b9-1a4f8a8640e9 service neutron] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:29:08.815948 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-862f77ff-9467-423c-89b9-1a4f8a8640e9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:29:08.827975 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e342b8e7-a870-4eeb-a2b7-070ec217f800 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:29:08.829636 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e342b8e7-a870-4eeb-a2b7-070ec217f800 service neutron] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:29:08.830513 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862f77ff-9467-423c-89b9-1a4f8a8640e9 service neutron] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:29:08.837295 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e7da0d-3207-43c6-992f-9100576536d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:29:08.838283 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e7da0d-3207-43c6-992f-9100576536d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:29:08.838641 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e7da0d-3207-43c6-992f-9100576536d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:29:08.838995 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e7da0d-3207-43c6-992f-9100576536d5 admin admin] PATH_INFO: `/v3/projects/ed11221969d14b2bb6ba403dd9c412a1` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:29:08.840232 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e7da0d-3207-43c6-992f-9100576536d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ed11221969d14b2bb6ba403dd9c412a1)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:29:08.841495 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1220/2437] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:29:08 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:29:08.842267 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e7da0d-3207-43c6-992f-9100576536d5 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:29:08.966430 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-02e7da0d-3207-43c6-992f-9100576536d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ed11221969d14b2bb6ba403dd9c412a1', 'request_id': 'req-02e7da0d-3207-43c6-992f-9100576536d5'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:29:08.974968 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-02e7da0d-3207-43c6-992f-9100576536d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ed11221969d14b2bb6ba403dd9c412a1', 'request_id': 'req-02e7da0d-3207-43c6-992f-9100576536d5'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:29:08.978953 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-02e7da0d-3207-43c6-992f-9100576536d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ed11221969d14b2bb6ba403dd9c412a1', 'request_id': 'req-02e7da0d-3207-43c6-992f-9100576536d5'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:29:09.058578 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-02e7da0d-3207-43c6-992f-9100576536d5 admin admin] The token cache is being invalidate because project ed11221969d14b2bb6ba403dd9c412a1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81749) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 01:29:09.059252 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-02e7da0d-3207-43c6-992f-9100576536d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:29:09.060150 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-02e7da0d-3207-43c6-992f-9100576536d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:29:09.066788 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1218/2438] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 01:29:08 2022] DELETE /identity/v3/projects/ed11221969d14b2bb6ba403dd9c412a1 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 01:29:09.066910 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab19c429-ff3a-4534-b385-8021327ddc78 None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:29:09.067028 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab19c429-ff3a-4534-b385-8021327ddc78 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:29:09.067296 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab19c429-ff3a-4534-b385-8021327ddc78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:29:09.089322 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-ab19c429-ff3a-4534-b385-8021327ddc78 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:29:09.090691 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-ab19c429-ff3a-4534-b385-8021327ddc78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:29:09.156495 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1221/2439] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:29:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:29:09.165836 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19db6529-8bf4-4676-ad81-275c3abf89bf None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:29:09.166133 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19db6529-8bf4-4676-ad81-275c3abf89bf None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:29:09.166508 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19db6529-8bf4-4676-ad81-275c3abf89bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:29:09.183156 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:29:09.189650 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-19db6529-8bf4-4676-ad81-275c3abf89bf None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:29:09.191113 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-19db6529-8bf4-4676-ad81-275c3abf89bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:29:09.200439 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1219/2440] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:29:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:29:09.205105 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab19c429-ff3a-4534-b385-8021327ddc78 None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:29:09.216175 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab19c429-ff3a-4534-b385-8021327ddc78 None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:29:09.221537 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b47660-51ba-4c02-b082-bc9dd91f5c9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:29:09.222275 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b47660-51ba-4c02-b082-bc9dd91f5c9d admin admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:29:09.222275 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b47660-51ba-4c02-b082-bc9dd91f5c9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:29:09.222469 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b47660-51ba-4c02-b082-bc9dd91f5c9d admin admin] PATH_INFO: `/v3/domains` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:29:09.222955 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b47660-51ba-4c02-b082-bc9dd91f5c9d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:29:09.223265 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b47660-51ba-4c02-b082-bc9dd91f5c9d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:29:09.224984 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b47660-51ba-4c02-b082-bc9dd91f5c9d admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:29:09.238072 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1222/2441] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Aug 31 01:29:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:29:09.244614 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19db6529-8bf4-4676-ad81-275c3abf89bf None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:29:09.255781 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19db6529-8bf4-4676-ad81-275c3abf89bf None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:29:09.261517 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc19e0b-1d0b-4e6b-a3b6-a7dda5a1c5a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:29:09.262172 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc19e0b-1d0b-4e6b-a3b6-a7dda5a1c5a6 admin admin] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:29:09.262463 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc19e0b-1d0b-4e6b-a3b6-a7dda5a1c5a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:29:09.262741 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc19e0b-1d0b-4e6b-a3b6-a7dda5a1c5a6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:29:09.263452 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc19e0b-1d0b-4e6b-a3b6-a7dda5a1c5a6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:29:09.265446 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc19e0b-1d0b-4e6b-a3b6-a7dda5a1c5a6 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:29:09.401228 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1220/2442] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 01:29:09 2022] POST /identity/v3/projects => generated 361 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:29:09.405330 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b47660-51ba-4c02-b082-bc9dd91f5c9d admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:29:09.416417 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b47660-51ba-4c02-b082-bc9dd91f5c9d admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:29:09.478253 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcfbb64-7a43-4097-90c9-ef8dcc85b678 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:29:09.478872 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcfbb64-7a43-4097-90c9-ef8dcc85b678 admin admin] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:29:09.479156 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcfbb64-7a43-4097-90c9-ef8dcc85b678 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:29:09.479567 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcfbb64-7a43-4097-90c9-ef8dcc85b678 admin admin] PATH_INFO: `/v3/users` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:29:09.480206 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcfbb64-7a43-4097-90c9-ef8dcc85b678 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:29:09.482180 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcfbb64-7a43-4097-90c9-ef8dcc85b678 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:29:09.643252 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1223/2443] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 01:29:09 2022] POST /identity/v3/users => generated 281 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:29:09.644396 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc19e0b-1d0b-4e6b-a3b6-a7dda5a1c5a6 admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:29:09.654827 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc19e0b-1d0b-4e6b-a3b6-a7dda5a1c5a6 admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:29:09.659794 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a564d49-1d98-4bb2-ba01-2a4e01efdbe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:29:09.660417 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a564d49-1d98-4bb2-ba01-2a4e01efdbe5 admin admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:29:09.660698 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a564d49-1d98-4bb2-ba01-2a4e01efdbe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:29:09.661054 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a564d49-1d98-4bb2-ba01-2a4e01efdbe5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:29:09.661706 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a564d49-1d98-4bb2-ba01-2a4e01efdbe5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:29:09.664148 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a564d49-1d98-4bb2-ba01-2a4e01efdbe5 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:29:09.681846 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1221/2444] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:29:09 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 01:29:09.686413 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcfbb64-7a43-4097-90c9-ef8dcc85b678 admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:29:09.697342 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcfbb64-7a43-4097-90c9-ef8dcc85b678 admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:29:09.702660 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6331c784-f38a-4561-bf46-38b95fa9a7f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:29:09.703351 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6331c784-f38a-4561-bf46-38b95fa9a7f7 admin admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:29:09.703980 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6331c784-f38a-4561-bf46-38b95fa9a7f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:29:09.703980 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6331c784-f38a-4561-bf46-38b95fa9a7f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:29:09.704678 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6331c784-f38a-4561-bf46-38b95fa9a7f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:29:09.707310 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6331c784-f38a-4561-bf46-38b95fa9a7f7 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:29:09.721523 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1224/2445] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:29:09 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:29:09.726654 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a564d49-1d98-4bb2-ba01-2a4e01efdbe5 admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:29:09.737439 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a564d49-1d98-4bb2-ba01-2a4e01efdbe5 admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:29:09.742265 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56302785-16d1-47ac-ac0a-8c09df0b8ab7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:29:09.742919 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56302785-16d1-47ac-ac0a-8c09df0b8ab7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:29:09.743203 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56302785-16d1-47ac-ac0a-8c09df0b8ab7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:29:09.743479 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56302785-16d1-47ac-ac0a-8c09df0b8ab7 admin admin] PATH_INFO: `/v3/projects/d0b3e1beac294000a7b414ff7b96b42c/users/d5988d69b6e849ec876268f9dfb9ecba/roles/d0b9768f4d014766b9326387f2e11f34` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:29:09.758384 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56302785-16d1-47ac-ac0a-8c09df0b8ab7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d0b3e1beac294000a7b414ff7b96b42c, user_id=d5988d69b6e849ec876268f9dfb9ecba, role_id=d0b9768f4d014766b9326387f2e11f34)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:29:09.761291 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56302785-16d1-47ac-ac0a-8c09df0b8ab7 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:29:09.863879 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1222/2446] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 01:29:09 2022] PUT /identity/v3/projects/d0b3e1beac294000a7b414ff7b96b42c/users/d5988d69b6e849ec876268f9dfb9ecba/roles/d0b9768f4d014766b9326387f2e11f34 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:29:09.874648 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19583bf8-535f-43b0-a489-d803f80dcea9 None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:29:09.878862 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19583bf8-535f-43b0-a489-d803f80dcea9 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:29:09.878862 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19583bf8-535f-43b0-a489-d803f80dcea9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:29:09.903691 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-19583bf8-535f-43b0-a489-d803f80dcea9 None None] MFA Rules not processed for user `d5988d69b6e849ec876268f9dfb9ecba`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:29:09.904958 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-19583bf8-535f-43b0-a489-d803f80dcea9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:29:09.936397 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:29:09.970385 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1225/2447] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:29:09 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:29:09.975688 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56302785-16d1-47ac-ac0a-8c09df0b8ab7 admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:29:10.038876 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56302785-16d1-47ac-ac0a-8c09df0b8ab7 admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:29:10.043567 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2a295a-14df-4c2c-a4a5-27fd293b285e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:29:10.044177 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2a295a-14df-4c2c-a4a5-27fd293b285e admin admin] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:29:10.044464 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2a295a-14df-4c2c-a4a5-27fd293b285e admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:29:10.044762 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2a295a-14df-4c2c-a4a5-27fd293b285e admin admin] PATH_INFO: `/v3/projects` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:29:10.045630 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2a295a-14df-4c2c-a4a5-27fd293b285e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:29:10.047584 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2a295a-14df-4c2c-a4a5-27fd293b285e admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:29:10.179118 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1223/2448] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 01:29:09 2022] POST /identity/v3/projects => generated 365 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:29:10.183214 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19583bf8-535f-43b0-a489-d803f80dcea9 None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:29:10.193824 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19583bf8-535f-43b0-a489-d803f80dcea9 None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:29:10.253724 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6dbd54c-9c80-4d5a-a587-1675d1d76200 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:29:10.254411 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dbd54c-9c80-4d5a-a587-1675d1d76200 admin admin] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:29:10.254651 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dbd54c-9c80-4d5a-a587-1675d1d76200 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:29:10.254916 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dbd54c-9c80-4d5a-a587-1675d1d76200 admin admin] PATH_INFO: `/v3/users` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:29:10.255550 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6dbd54c-9c80-4d5a-a587-1675d1d76200 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:29:10.257609 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6dbd54c-9c80-4d5a-a587-1675d1d76200 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:29:10.373736 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1226/2449] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 01:29:10 2022] POST /identity/v3/users => generated 289 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:29:10.377786 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2a295a-14df-4c2c-a4a5-27fd293b285e admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:29:10.388246 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2a295a-14df-4c2c-a4a5-27fd293b285e admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:29:10.399076 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d36f917-e3d5-4acc-951e-3359c34549de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:29:10.400165 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d36f917-e3d5-4acc-951e-3359c34549de admin admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:29:10.400458 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d36f917-e3d5-4acc-951e-3359c34549de admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:29:10.400753 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d36f917-e3d5-4acc-951e-3359c34549de admin admin] PATH_INFO: `/v3/roles` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:29:10.401744 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d36f917-e3d5-4acc-951e-3359c34549de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:29:10.403019 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d36f917-e3d5-4acc-951e-3359c34549de admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:29:10.416137 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1224/2450] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:29:10 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:29:10.422142 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6dbd54c-9c80-4d5a-a587-1675d1d76200 admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:29:10.433945 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6dbd54c-9c80-4d5a-a587-1675d1d76200 admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:29:10.438805 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf024e0c-7a6c-4527-91a6-1c316812ebfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:29:10.439499 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf024e0c-7a6c-4527-91a6-1c316812ebfd admin admin] REQUEST_METHOD: `PUT` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:29:10.439781 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf024e0c-7a6c-4527-91a6-1c316812ebfd admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:29:10.440059 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf024e0c-7a6c-4527-91a6-1c316812ebfd admin admin] PATH_INFO: `/v3/projects/6dd9dc928b70487590a2fb0d50b8eaa3/users/a08c7f4d0fb047db9f5a82f1efca325f/roles/d009411496b8452b88e77ffb8db015a7` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:29:10.455604 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf024e0c-7a6c-4527-91a6-1c316812ebfd admin admin] RBAC: Authorizing `identity:create_grant(project_id=6dd9dc928b70487590a2fb0d50b8eaa3, user_id=a08c7f4d0fb047db9f5a82f1efca325f, role_id=d009411496b8452b88e77ffb8db015a7)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:29:10.458297 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf024e0c-7a6c-4527-91a6-1c316812ebfd admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:29:10.528231 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1227/2451] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 01:29:10 2022] PUT /identity/v3/projects/6dd9dc928b70487590a2fb0d50b8eaa3/users/a08c7f4d0fb047db9f5a82f1efca325f/roles/d009411496b8452b88e77ffb8db015a7 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:29:10.532504 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d36f917-e3d5-4acc-951e-3359c34549de admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:29:10.543694 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d36f917-e3d5-4acc-951e-3359c34549de admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:29:10.605991 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f120265-b37d-4cd6-b78b-93aee897d474 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:29:10.606684 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f120265-b37d-4cd6-b78b-93aee897d474 admin admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:29:10.617085 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f120265-b37d-4cd6-b78b-93aee897d474 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:29:10.617085 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f120265-b37d-4cd6-b78b-93aee897d474 admin admin] PATH_INFO: `/v3/roles` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:29:10.617085 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f120265-b37d-4cd6-b78b-93aee897d474 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:29:10.617085 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f120265-b37d-4cd6-b78b-93aee897d474 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:29:10.620596 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1225/2452] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:29:10 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:29:10.624962 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf024e0c-7a6c-4527-91a6-1c316812ebfd admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:29:10.636714 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf024e0c-7a6c-4527-91a6-1c316812ebfd admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:29:10.643102 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5164d963-4d67-43cd-838f-cda7535baa57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:29:10.643902 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5164d963-4d67-43cd-838f-cda7535baa57 admin admin] REQUEST_METHOD: `PUT` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:29:10.644363 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5164d963-4d67-43cd-838f-cda7535baa57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:29:10.644600 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5164d963-4d67-43cd-838f-cda7535baa57 admin admin] PATH_INFO: `/v3/projects/6dd9dc928b70487590a2fb0d50b8eaa3/users/a08c7f4d0fb047db9f5a82f1efca325f/roles/d009411496b8452b88e77ffb8db015a7` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:29:10.647559 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5164d963-4d67-43cd-838f-cda7535baa57 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6dd9dc928b70487590a2fb0d50b8eaa3, user_id=a08c7f4d0fb047db9f5a82f1efca325f, role_id=d009411496b8452b88e77ffb8db015a7)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:29:10.650526 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5164d963-4d67-43cd-838f-cda7535baa57 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:29:10.667630 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1228/2453] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 01:29:10 2022] PUT /identity/v3/projects/6dd9dc928b70487590a2fb0d50b8eaa3/users/a08c7f4d0fb047db9f5a82f1efca325f/roles/d009411496b8452b88e77ffb8db015a7 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:29:10.685175 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b5af19-a4c6-40fa-ba45-8b96fd90544e None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:29:10.685479 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b5af19-a4c6-40fa-ba45-8b96fd90544e None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:29:10.685826 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b5af19-a4c6-40fa-ba45-8b96fd90544e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:29:10.715113 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:29:10.718970 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-07b5af19-a4c6-40fa-ba45-8b96fd90544e None None] MFA Rules not processed for user `a08c7f4d0fb047db9f5a82f1efca325f`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:29:10.720259 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-07b5af19-a4c6-40fa-ba45-8b96fd90544e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:29:10.813224 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1226/2454] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:29:10 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:29:10.826373 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5164d963-4d67-43cd-838f-cda7535baa57 admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:29:10.828114 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-5164d963-4d67-43cd-838f-cda7535baa57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:29:10.920235 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5164d963-4d67-43cd-838f-cda7535baa57 admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:29:10.926528 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef081fb6-4725-4094-9e53-6db7ab286e34 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5c8fc69c518412ba1aad61a9f4f558c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:29:10.927147 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef081fb6-4725-4094-9e53-6db7ab286e34 service cinder] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:29:10.927432 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef081fb6-4725-4094-9e53-6db7ab286e34 service cinder] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:29:10.927707 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef081fb6-4725-4094-9e53-6db7ab286e34 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:29:10.937987 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef081fb6-4725-4094-9e53-6db7ab286e34 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:29:10.940757 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef081fb6-4725-4094-9e53-6db7ab286e34 service cinder] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:29:10.955022 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1229/2455] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:29:10 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:29:11.137777 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b5af19-a4c6-40fa-ba45-8b96fd90544e None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:29:11.150278 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-07b5af19-a4c6-40fa-ba45-8b96fd90544e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:29:11.155814 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b5af19-a4c6-40fa-ba45-8b96fd90544e None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:29:11.160427 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db13343-9cf7-4bed-b1d5-6b3f5f3b9359 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5c8fc69c518412ba1aad61a9f4f558c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:29:11.161068 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db13343-9cf7-4bed-b1d5-6b3f5f3b9359 service cinder] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:29:11.161675 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db13343-9cf7-4bed-b1d5-6b3f5f3b9359 service cinder] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:29:11.162318 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db13343-9cf7-4bed-b1d5-6b3f5f3b9359 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:29:11.250662 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db13343-9cf7-4bed-b1d5-6b3f5f3b9359 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:29:11.252643 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db13343-9cf7-4bed-b1d5-6b3f5f3b9359 service cinder] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:29:11.263733 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1227/2456] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:29:11 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:29:14.853562 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef081fb6-4725-4094-9e53-6db7ab286e34 service cinder] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:29:14.931912 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:29:14.986906 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef081fb6-4725-4094-9e53-6db7ab286e34 service cinder] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:29:14.992828 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3347453c-c581-45f1-a5a2-51a39ca2baf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:29:14.993484 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3347453c-c581-45f1-a5a2-51a39ca2baf7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:29:14.993774 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3347453c-c581-45f1-a5a2-51a39ca2baf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:29:14.994058 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3347453c-c581-45f1-a5a2-51a39ca2baf7 admin admin] PATH_INFO: `/v3/users/d5988d69b6e849ec876268f9dfb9ecba` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:29:14.995229 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3347453c-c581-45f1-a5a2-51a39ca2baf7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d5988d69b6e849ec876268f9dfb9ecba)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:29:14.997361 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3347453c-c581-45f1-a5a2-51a39ca2baf7 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:29:15.195562 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-3347453c-c581-45f1-a5a2-51a39ca2baf7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd5988d69b6e849ec876268f9dfb9ecba'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:29:15.225623 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-3347453c-c581-45f1-a5a2-51a39ca2baf7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd5988d69b6e849ec876268f9dfb9ecba'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:29:15.226584 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-3347453c-c581-45f1-a5a2-51a39ca2baf7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd5988d69b6e849ec876268f9dfb9ecba'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:29:15.238810 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-3347453c-c581-45f1-a5a2-51a39ca2baf7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd5988d69b6e849ec876268f9dfb9ecba'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:29:15.306689 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-3347453c-c581-45f1-a5a2-51a39ca2baf7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd5988d69b6e849ec876268f9dfb9ecba'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:29:15.335127 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1230/2457] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 01:29:14 2022] DELETE /identity/v3/users/d5988d69b6e849ec876268f9dfb9ecba => generated 0 bytes in 481 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:29:15.339415 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f699d6cd-10dc-4809-9f2e-e6809a186a75 None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:29:15.343260 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f699d6cd-10dc-4809-9f2e-e6809a186a75 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:29:15.343260 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f699d6cd-10dc-4809-9f2e-e6809a186a75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:29:15.372265 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-f699d6cd-10dc-4809-9f2e-e6809a186a75 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:29:15.374068 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-f699d6cd-10dc-4809-9f2e-e6809a186a75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:29:15.498680 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1228/2458] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:29:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:29:15.526745 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3347453c-c581-45f1-a5a2-51a39ca2baf7 admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:29:15.526745 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-3347453c-c581-45f1-a5a2-51a39ca2baf7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:29:15.591802 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3347453c-c581-45f1-a5a2-51a39ca2baf7 admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:29:15.596935 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8747624e-1665-4b30-abf8-b582f8fbd654 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0817c3320d5d41d8bba0550c8ae8dd60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:29:15.597578 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8747624e-1665-4b30-abf8-b582f8fbd654 service neutron] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:29:15.597864 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8747624e-1665-4b30-abf8-b582f8fbd654 service neutron] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:29:15.598389 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8747624e-1665-4b30-abf8-b582f8fbd654 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:29:15.612987 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8747624e-1665-4b30-abf8-b582f8fbd654 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:29:15.614448 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8747624e-1665-4b30-abf8-b582f8fbd654 service neutron] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:29:15.627092 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1231/2459] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:29:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 01:29:16.047628 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f699d6cd-10dc-4809-9f2e-e6809a186a75 None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:29:16.047628 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-f699d6cd-10dc-4809-9f2e-e6809a186a75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:29:16.079146 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f699d6cd-10dc-4809-9f2e-e6809a186a75 None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:29:16.084682 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3647a1-1ef7-4a7f-a3a0-4f16ab79f34d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:29:16.085355 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3647a1-1ef7-4a7f-a3a0-4f16ab79f34d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:29:16.085640 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3647a1-1ef7-4a7f-a3a0-4f16ab79f34d admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:29:16.085975 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3647a1-1ef7-4a7f-a3a0-4f16ab79f34d admin admin] PATH_INFO: `/v3/projects/d0b3e1beac294000a7b414ff7b96b42c` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:29:16.087121 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3647a1-1ef7-4a7f-a3a0-4f16ab79f34d admin admin] RBAC: Authorizing `identity:delete_project(project_id=d0b3e1beac294000a7b414ff7b96b42c)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:29:16.089176 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3647a1-1ef7-4a7f-a3a0-4f16ab79f34d admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:29:16.162848 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:29:16.175724 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-df3647a1-1ef7-4a7f-a3a0-4f16ab79f34d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd0b3e1beac294000a7b414ff7b96b42c', 'request_id': 'req-df3647a1-1ef7-4a7f-a3a0-4f16ab79f34d'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:29:16.182608 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-df3647a1-1ef7-4a7f-a3a0-4f16ab79f34d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd0b3e1beac294000a7b414ff7b96b42c', 'request_id': 'req-df3647a1-1ef7-4a7f-a3a0-4f16ab79f34d'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:29:16.190779 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-df3647a1-1ef7-4a7f-a3a0-4f16ab79f34d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd0b3e1beac294000a7b414ff7b96b42c', 'request_id': 'req-df3647a1-1ef7-4a7f-a3a0-4f16ab79f34d'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:29:16.267692 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-df3647a1-1ef7-4a7f-a3a0-4f16ab79f34d admin admin] The token cache is being invalidate because project d0b3e1beac294000a7b414ff7b96b42c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81749) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 01:29:16.267990 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-df3647a1-1ef7-4a7f-a3a0-4f16ab79f34d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:29:16.268618 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-df3647a1-1ef7-4a7f-a3a0-4f16ab79f34d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:29:16.270236 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1229/2460] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 01:29:16 2022] DELETE /identity/v3/projects/d0b3e1beac294000a7b414ff7b96b42c => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:29:16.274877 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8747624e-1665-4b30-abf8-b582f8fbd654 service neutron] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:29:16.284836 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8747624e-1665-4b30-abf8-b582f8fbd654 service neutron] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:29:16.330767 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:29:16.343958 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdd6d47-e432-4dd2-96ec-8ea26e1032cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:29:16.344588 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdd6d47-e432-4dd2-96ec-8ea26e1032cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:29:16.344903 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdd6d47-e432-4dd2-96ec-8ea26e1032cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:29:16.345186 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdd6d47-e432-4dd2-96ec-8ea26e1032cc admin admin] PATH_INFO: `/v3/users/a08c7f4d0fb047db9f5a82f1efca325f` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:29:16.346325 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdd6d47-e432-4dd2-96ec-8ea26e1032cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=a08c7f4d0fb047db9f5a82f1efca325f)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:29:16.348327 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdd6d47-e432-4dd2-96ec-8ea26e1032cc admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:29:16.556260 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-0bdd6d47-e432-4dd2-96ec-8ea26e1032cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a08c7f4d0fb047db9f5a82f1efca325f'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:29:16.569196 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-0bdd6d47-e432-4dd2-96ec-8ea26e1032cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a08c7f4d0fb047db9f5a82f1efca325f'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:29:16.569953 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-0bdd6d47-e432-4dd2-96ec-8ea26e1032cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a08c7f4d0fb047db9f5a82f1efca325f'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:29:16.577326 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-0bdd6d47-e432-4dd2-96ec-8ea26e1032cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a08c7f4d0fb047db9f5a82f1efca325f'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:29:16.633897 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-0bdd6d47-e432-4dd2-96ec-8ea26e1032cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a08c7f4d0fb047db9f5a82f1efca325f'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:29:16.640660 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1232/2461] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 01:29:16 2022] DELETE /identity/v3/users/a08c7f4d0fb047db9f5a82f1efca325f => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:29:17.166730 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3647a1-1ef7-4a7f-a3a0-4f16ab79f34d admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:29:17.168421 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-df3647a1-1ef7-4a7f-a3a0-4f16ab79f34d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:29:17.183778 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3647a1-1ef7-4a7f-a3a0-4f16ab79f34d admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:29:17.245753 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5add0d-6be8-4bf0-9086-d16f88782b09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:29:17.246393 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5add0d-6be8-4bf0-9086-d16f88782b09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:29:17.246678 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5add0d-6be8-4bf0-9086-d16f88782b09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:29:17.246960 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5add0d-6be8-4bf0-9086-d16f88782b09 admin admin] PATH_INFO: `/v3/projects/6dd9dc928b70487590a2fb0d50b8eaa3` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:29:17.248079 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5add0d-6be8-4bf0-9086-d16f88782b09 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6dd9dc928b70487590a2fb0d50b8eaa3)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:29:17.250355 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5add0d-6be8-4bf0-9086-d16f88782b09 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:29:17.360062 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-fc5add0d-6be8-4bf0-9086-d16f88782b09 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6dd9dc928b70487590a2fb0d50b8eaa3', 'request_id': 'req-fc5add0d-6be8-4bf0-9086-d16f88782b09'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:29:17.365735 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-fc5add0d-6be8-4bf0-9086-d16f88782b09 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6dd9dc928b70487590a2fb0d50b8eaa3', 'request_id': 'req-fc5add0d-6be8-4bf0-9086-d16f88782b09'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:29:17.372790 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-fc5add0d-6be8-4bf0-9086-d16f88782b09 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6dd9dc928b70487590a2fb0d50b8eaa3', 'request_id': 'req-fc5add0d-6be8-4bf0-9086-d16f88782b09'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:29:17.451405 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-fc5add0d-6be8-4bf0-9086-d16f88782b09 admin admin] The token cache is being invalidate because project 6dd9dc928b70487590a2fb0d50b8eaa3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81749) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 01:29:17.451719 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-fc5add0d-6be8-4bf0-9086-d16f88782b09 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:29:17.452402 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-fc5add0d-6be8-4bf0-9086-d16f88782b09 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:29:17.454032 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1230/2462] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 01:29:17 2022] DELETE /identity/v3/projects/6dd9dc928b70487590a2fb0d50b8eaa3 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:29:17.459907 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fec27f-1ce0-49cd-bfd2-f427f34adb70 None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:29:17.460366 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fec27f-1ce0-49cd-bfd2-f427f34adb70 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:29:17.460450 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fec27f-1ce0-49cd-bfd2-f427f34adb70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:29:17.482822 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-c9fec27f-1ce0-49cd-bfd2-f427f34adb70 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:29:17.484199 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-c9fec27f-1ce0-49cd-bfd2-f427f34adb70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:29:17.544220 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1233/2463] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:29:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:29:17.552157 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abd4d03-0f79-4c60-bc33-7d0d2c5930b3 None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:29:17.552474 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abd4d03-0f79-4c60-bc33-7d0d2c5930b3 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:29:17.552918 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abd4d03-0f79-4c60-bc33-7d0d2c5930b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:29:17.573014 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-3abd4d03-0f79-4c60-bc33-7d0d2c5930b3 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:29:17.574227 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-3abd4d03-0f79-4c60-bc33-7d0d2c5930b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:29:17.582704 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1231/2464] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:29:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:29:17.587148 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fec27f-1ce0-49cd-bfd2-f427f34adb70 None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:29:17.597041 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fec27f-1ce0-49cd-bfd2-f427f34adb70 None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:29:17.602075 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52a4812-9bee-4c4d-babe-7bdce3a78cb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:29:17.602661 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52a4812-9bee-4c4d-babe-7bdce3a78cb6 admin admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:29:17.602940 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52a4812-9bee-4c4d-babe-7bdce3a78cb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:29:17.603211 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52a4812-9bee-4c4d-babe-7bdce3a78cb6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:29:17.603841 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52a4812-9bee-4c4d-babe-7bdce3a78cb6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:29:17.604138 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52a4812-9bee-4c4d-babe-7bdce3a78cb6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:29:17.605755 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52a4812-9bee-4c4d-babe-7bdce3a78cb6 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:29:17.618076 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1234/2465] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Aug 31 01:29:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:29:17.622275 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abd4d03-0f79-4c60-bc33-7d0d2c5930b3 None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:29:17.631295 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abd4d03-0f79-4c60-bc33-7d0d2c5930b3 None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:29:17.635880 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6a496d-fa8b-459d-bf94-0884bf0c428a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:29:17.636466 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6a496d-fa8b-459d-bf94-0884bf0c428a admin admin] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:29:17.636881 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6a496d-fa8b-459d-bf94-0884bf0c428a admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:29:17.637157 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6a496d-fa8b-459d-bf94-0884bf0c428a admin admin] PATH_INFO: `/v3/projects` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:29:17.637779 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6a496d-fa8b-459d-bf94-0884bf0c428a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:29:17.639733 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6a496d-fa8b-459d-bf94-0884bf0c428a admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:29:17.746692 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1232/2466] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 01:29:17 2022] POST /identity/v3/projects => generated 377 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 01:29:17.750838 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52a4812-9bee-4c4d-babe-7bdce3a78cb6 admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:29:17.760951 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52a4812-9bee-4c4d-babe-7bdce3a78cb6 admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:29:17.810365 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:29:17.817295 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6cd5d1-9cd0-4b7f-9e93-d6d82abdf552 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:29:17.817934 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6cd5d1-9cd0-4b7f-9e93-d6d82abdf552 admin admin] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:29:17.818220 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6cd5d1-9cd0-4b7f-9e93-d6d82abdf552 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:29:17.818522 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6cd5d1-9cd0-4b7f-9e93-d6d82abdf552 admin admin] PATH_INFO: `/v3/users` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:29:17.819254 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6cd5d1-9cd0-4b7f-9e93-d6d82abdf552 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:29:17.821278 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6cd5d1-9cd0-4b7f-9e93-d6d82abdf552 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:29:17.908054 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1235/2467] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 01:29:17 2022] POST /identity/v3/users => generated 289 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:29:17.912881 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6a496d-fa8b-459d-bf94-0884bf0c428a admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:29:17.923300 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6a496d-fa8b-459d-bf94-0884bf0c428a admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:29:17.928290 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fa721b-6c16-42e4-9533-ca1c3f009bf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:29:17.929150 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fa721b-6c16-42e4-9533-ca1c3f009bf2 admin admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:29:17.929591 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fa721b-6c16-42e4-9533-ca1c3f009bf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:29:17.930105 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fa721b-6c16-42e4-9533-ca1c3f009bf2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:29:17.931070 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fa721b-6c16-42e4-9533-ca1c3f009bf2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:29:17.933878 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fa721b-6c16-42e4-9533-ca1c3f009bf2 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:29:17.943889 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1233/2468] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:29:17 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:29:17.947980 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6cd5d1-9cd0-4b7f-9e93-d6d82abdf552 admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:29:17.958365 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6cd5d1-9cd0-4b7f-9e93-d6d82abdf552 admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:29:17.963520 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f5c038-46c4-42d0-9fa9-21fcd84805f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:29:17.964185 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f5c038-46c4-42d0-9fa9-21fcd84805f7 admin admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:29:17.964466 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f5c038-46c4-42d0-9fa9-21fcd84805f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:29:17.964774 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f5c038-46c4-42d0-9fa9-21fcd84805f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:29:17.965413 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f5c038-46c4-42d0-9fa9-21fcd84805f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:29:17.967005 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f5c038-46c4-42d0-9fa9-21fcd84805f7 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:29:17.978199 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1236/2469] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:29:17 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:29:17.982239 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fa721b-6c16-42e4-9533-ca1c3f009bf2 admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:29:17.992139 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fa721b-6c16-42e4-9533-ca1c3f009bf2 admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:29:17.996706 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8058aed7-127f-4f4d-822d-b4298d055dcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:29:17.999683 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8058aed7-127f-4f4d-822d-b4298d055dcc admin admin] REQUEST_METHOD: `PUT` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:29:17.999683 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8058aed7-127f-4f4d-822d-b4298d055dcc admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:29:17.999683 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8058aed7-127f-4f4d-822d-b4298d055dcc admin admin] PATH_INFO: `/v3/projects/bb74d110d646466794ccb97d204e1480/users/d963744868964b399657c16edd212111/roles/d0b9768f4d014766b9326387f2e11f34` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:29:18.006261 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:29:18.012746 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8058aed7-127f-4f4d-822d-b4298d055dcc admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb74d110d646466794ccb97d204e1480, user_id=d963744868964b399657c16edd212111, role_id=d0b9768f4d014766b9326387f2e11f34)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:29:18.015471 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8058aed7-127f-4f4d-822d-b4298d055dcc admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:29:18.096965 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1234/2470] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 01:29:17 2022] PUT /identity/v3/projects/bb74d110d646466794ccb97d204e1480/users/d963744868964b399657c16edd212111/roles/d0b9768f4d014766b9326387f2e11f34 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:29:18.107881 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e612cfe4-8964-4a52-9ecd-eb314748facd None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:29:18.108172 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e612cfe4-8964-4a52-9ecd-eb314748facd None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:29:18.108451 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e612cfe4-8964-4a52-9ecd-eb314748facd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:29:18.136261 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-e612cfe4-8964-4a52-9ecd-eb314748facd None None] MFA Rules not processed for user `d963744868964b399657c16edd212111`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:29:18.137571 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-e612cfe4-8964-4a52-9ecd-eb314748facd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:29:18.205150 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1237/2471] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:29:18 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:29:18.210118 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8058aed7-127f-4f4d-822d-b4298d055dcc admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:29:18.270187 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8058aed7-127f-4f4d-822d-b4298d055dcc admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:29:18.275512 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2244abf0-d589-4dd0-8efe-1cae4c12dc40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:29:18.276121 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2244abf0-d589-4dd0-8efe-1cae4c12dc40 admin admin] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:29:18.276406 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2244abf0-d589-4dd0-8efe-1cae4c12dc40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:29:18.276684 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2244abf0-d589-4dd0-8efe-1cae4c12dc40 admin admin] PATH_INFO: `/v3/projects` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:29:18.277348 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2244abf0-d589-4dd0-8efe-1cae4c12dc40 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:29:18.279266 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2244abf0-d589-4dd0-8efe-1cae4c12dc40 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:29:18.368821 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1235/2472] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 01:29:18 2022] POST /identity/v3/projects => generated 377 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 01:29:18.371017 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e612cfe4-8964-4a52-9ecd-eb314748facd None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:29:18.382118 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e612cfe4-8964-4a52-9ecd-eb314748facd None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:29:18.439217 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284e6a29-e315-4a31-8b57-78f0d390ab5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:29:18.439754 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284e6a29-e315-4a31-8b57-78f0d390ab5f admin admin] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:29:18.440044 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284e6a29-e315-4a31-8b57-78f0d390ab5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:29:18.440323 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284e6a29-e315-4a31-8b57-78f0d390ab5f admin admin] PATH_INFO: `/v3/users` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:29:18.441007 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284e6a29-e315-4a31-8b57-78f0d390ab5f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:29:18.442958 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284e6a29-e315-4a31-8b57-78f0d390ab5f admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:29:18.548238 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1238/2473] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 01:29:18 2022] POST /identity/v3/users => generated 295 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:29:18.552852 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2244abf0-d589-4dd0-8efe-1cae4c12dc40 admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:29:18.563067 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2244abf0-d589-4dd0-8efe-1cae4c12dc40 admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:29:18.567630 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cf93bd-84b4-4460-882e-192dbfc994cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:29:18.568709 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cf93bd-84b4-4460-882e-192dbfc994cc admin admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:29:18.569016 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cf93bd-84b4-4460-882e-192dbfc994cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:29:18.569287 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cf93bd-84b4-4460-882e-192dbfc994cc admin admin] PATH_INFO: `/v3/roles` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:29:18.569918 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cf93bd-84b4-4460-882e-192dbfc994cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:29:18.571520 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cf93bd-84b4-4460-882e-192dbfc994cc admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:29:18.582435 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1236/2474] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:29:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:29:18.587606 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284e6a29-e315-4a31-8b57-78f0d390ab5f admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:29:18.597676 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284e6a29-e315-4a31-8b57-78f0d390ab5f admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:29:18.602387 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55b0568-493a-4877-8ba0-f8fda9c8fd2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:29:18.603030 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55b0568-493a-4877-8ba0-f8fda9c8fd2c admin admin] REQUEST_METHOD: `PUT` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:29:18.603320 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55b0568-493a-4877-8ba0-f8fda9c8fd2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:29:18.603721 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55b0568-493a-4877-8ba0-f8fda9c8fd2c admin admin] PATH_INFO: `/v3/projects/d2cbab83125a4121b2d4f1ca6a987373/users/3351f082868b4ff2b9688cdd49dcbdc0/roles/d009411496b8452b88e77ffb8db015a7` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:29:18.618766 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55b0568-493a-4877-8ba0-f8fda9c8fd2c admin admin] RBAC: Authorizing `identity:create_grant(project_id=d2cbab83125a4121b2d4f1ca6a987373, user_id=3351f082868b4ff2b9688cdd49dcbdc0, role_id=d009411496b8452b88e77ffb8db015a7)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:29:18.621456 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55b0568-493a-4877-8ba0-f8fda9c8fd2c admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:29:18.673052 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1239/2475] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 01:29:18 2022] PUT /identity/v3/projects/d2cbab83125a4121b2d4f1ca6a987373/users/3351f082868b4ff2b9688cdd49dcbdc0/roles/d009411496b8452b88e77ffb8db015a7 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:29:18.677055 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cf93bd-84b4-4460-882e-192dbfc994cc admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:29:18.687714 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cf93bd-84b4-4460-882e-192dbfc994cc admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:29:18.749003 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea1605d-c26a-4219-924b-79d7163b4568 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:29:18.749609 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea1605d-c26a-4219-924b-79d7163b4568 admin admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:29:18.749896 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea1605d-c26a-4219-924b-79d7163b4568 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:29:18.750166 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea1605d-c26a-4219-924b-79d7163b4568 admin admin] PATH_INFO: `/v3/roles` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:29:18.750804 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea1605d-c26a-4219-924b-79d7163b4568 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:29:18.752386 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea1605d-c26a-4219-924b-79d7163b4568 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:29:18.762718 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1237/2476] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:29:18 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:29:18.766933 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55b0568-493a-4877-8ba0-f8fda9c8fd2c admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:29:18.777271 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55b0568-493a-4877-8ba0-f8fda9c8fd2c admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:29:18.781937 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171b3bd2-6378-4c19-965b-aca6b52461a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:29:18.782581 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171b3bd2-6378-4c19-965b-aca6b52461a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:29:18.782863 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171b3bd2-6378-4c19-965b-aca6b52461a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:29:18.783133 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171b3bd2-6378-4c19-965b-aca6b52461a3 admin admin] PATH_INFO: `/v3/projects/d2cbab83125a4121b2d4f1ca6a987373/users/3351f082868b4ff2b9688cdd49dcbdc0/roles/d009411496b8452b88e77ffb8db015a7` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:29:18.785070 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171b3bd2-6378-4c19-965b-aca6b52461a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d2cbab83125a4121b2d4f1ca6a987373, user_id=3351f082868b4ff2b9688cdd49dcbdc0, role_id=d009411496b8452b88e77ffb8db015a7)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:29:18.787778 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171b3bd2-6378-4c19-965b-aca6b52461a3 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:29:18.797795 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1240/2477] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 01:29:18 2022] PUT /identity/v3/projects/d2cbab83125a4121b2d4f1ca6a987373/users/3351f082868b4ff2b9688cdd49dcbdc0/roles/d009411496b8452b88e77ffb8db015a7 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:29:18.809424 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2982b6-262f-49a1-8e4e-7a1a708797fa None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:29:18.809751 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2982b6-262f-49a1-8e4e-7a1a708797fa None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:29:18.810095 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2982b6-262f-49a1-8e4e-7a1a708797fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:29:18.838939 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-fa2982b6-262f-49a1-8e4e-7a1a708797fa None None] MFA Rules not processed for user `3351f082868b4ff2b9688cdd49dcbdc0`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:29:18.840196 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-fa2982b6-262f-49a1-8e4e-7a1a708797fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:29:18.890152 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:29:18.920418 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1238/2478] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:29:18 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:29:18.931692 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171b3bd2-6378-4c19-965b-aca6b52461a3 admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:29:18.933618 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-171b3bd2-6378-4c19-965b-aca6b52461a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:29:18.954411 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:29:18.992373 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171b3bd2-6378-4c19-965b-aca6b52461a3 admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:29:18.996145 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c9383a-d785-4dd1-9126-54a682db10dc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5c8fc69c518412ba1aad61a9f4f558c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:29:18.996772 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c9383a-d785-4dd1-9126-54a682db10dc service cinder] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:29:18.997055 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c9383a-d785-4dd1-9126-54a682db10dc service cinder] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:29:18.997329 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c9383a-d785-4dd1-9126-54a682db10dc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:29:19.057385 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c9383a-d785-4dd1-9126-54a682db10dc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:29:19.058787 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c9383a-d785-4dd1-9126-54a682db10dc service cinder] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:29:19.067621 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1241/2479] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:29:18 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:29:21.452397 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2982b6-262f-49a1-8e4e-7a1a708797fa None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:29:21.453831 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-fa2982b6-262f-49a1-8e4e-7a1a708797fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:29:21.511635 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2982b6-262f-49a1-8e4e-7a1a708797fa None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:29:21.515280 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5706ca44-c97a-496d-923f-320a2be97f96 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0817c3320d5d41d8bba0550c8ae8dd60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:29:21.515882 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5706ca44-c97a-496d-923f-320a2be97f96 service neutron] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:29:21.516163 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5706ca44-c97a-496d-923f-320a2be97f96 service neutron] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:29:21.516439 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5706ca44-c97a-496d-923f-320a2be97f96 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:29:21.518310 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-5706ca44-c97a-496d-923f-320a2be97f96 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:29:21.579554 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5706ca44-c97a-496d-923f-320a2be97f96 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:29:21.581371 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5706ca44-c97a-496d-923f-320a2be97f96 service neutron] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:29:21.591586 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1239/2480] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:29:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:29:25.085808 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c9383a-d785-4dd1-9126-54a682db10dc service cinder] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:29:25.087520 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-a3c9383a-d785-4dd1-9126-54a682db10dc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:29:25.166891 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c9383a-d785-4dd1-9126-54a682db10dc service cinder] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:29:25.171955 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb35087-0a06-4f22-ba99-7ea8156583ed service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dea0fe5b1d6b405cb47b95192580303b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:29:25.172506 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb35087-0a06-4f22-ba99-7ea8156583ed service placement] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:29:25.172829 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb35087-0a06-4f22-ba99-7ea8156583ed service placement] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:29:25.173109 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb35087-0a06-4f22-ba99-7ea8156583ed service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:29:25.174876 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-abb35087-0a06-4f22-ba99-7ea8156583ed service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:29:25.188759 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb35087-0a06-4f22-ba99-7ea8156583ed service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:29:25.190234 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb35087-0a06-4f22-ba99-7ea8156583ed service placement] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:29:25.201636 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1242/2481] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:29:25 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:29:25.252945 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5706ca44-c97a-496d-923f-320a2be97f96 service neutron] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:29:25.254792 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-5706ca44-c97a-496d-923f-320a2be97f96 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:29:25.269272 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5706ca44-c97a-496d-923f-320a2be97f96 service neutron] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:29:25.275229 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c70d21-0bde-4bf4-b0c4-b43d51371100 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:29:25.275887 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c70d21-0bde-4bf4-b0c4-b43d51371100 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:29:25.276177 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c70d21-0bde-4bf4-b0c4-b43d51371100 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:29:25.276529 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c70d21-0bde-4bf4-b0c4-b43d51371100 admin admin] PATH_INFO: `/v3/users/1cc3db55df284cd39f28fcfefc619fda` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:29:25.277800 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c70d21-0bde-4bf4-b0c4-b43d51371100 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1cc3db55df284cd39f28fcfefc619fda)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:29:25.279869 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c70d21-0bde-4bf4-b0c4-b43d51371100 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:29:25.447523 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-e0c70d21-0bde-4bf4-b0c4-b43d51371100 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1cc3db55df284cd39f28fcfefc619fda'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:29:25.501707 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb35087-0a06-4f22-ba99-7ea8156583ed service placement] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:29:25.503110 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-abb35087-0a06-4f22-ba99-7ea8156583ed service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:29:25.515392 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb35087-0a06-4f22-ba99-7ea8156583ed service placement] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:29:25.519104 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e80631f-98f2-4efc-9e27-c3d6f75e78b2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5c8fc69c518412ba1aad61a9f4f558c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:29:25.519701 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e80631f-98f2-4efc-9e27-c3d6f75e78b2 service cinder] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:29:25.519987 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e80631f-98f2-4efc-9e27-c3d6f75e78b2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:29:25.520259 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e80631f-98f2-4efc-9e27-c3d6f75e78b2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:29:25.529179 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e80631f-98f2-4efc-9e27-c3d6f75e78b2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:29:25.530604 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e80631f-98f2-4efc-9e27-c3d6f75e78b2 service cinder] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:29:25.536918 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-e0c70d21-0bde-4bf4-b0c4-b43d51371100 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1cc3db55df284cd39f28fcfefc619fda'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:29:25.543396 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-e0c70d21-0bde-4bf4-b0c4-b43d51371100 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1cc3db55df284cd39f28fcfefc619fda'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:29:25.543526 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1243/2482] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:29:25 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 01:29:25.545722 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-e0c70d21-0bde-4bf4-b0c4-b43d51371100 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1cc3db55df284cd39f28fcfefc619fda'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:29:25.546532 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-e0c70d21-0bde-4bf4-b0c4-b43d51371100 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1cc3db55df284cd39f28fcfefc619fda'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:29:25.565120 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1240/2483] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 01:29:25 2022] DELETE /identity/v3/users/1cc3db55df284cd39f28fcfefc619fda => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:29:25.863946 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e80631f-98f2-4efc-9e27-c3d6f75e78b2 service cinder] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:29:25.877440 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e80631f-98f2-4efc-9e27-c3d6f75e78b2 service cinder] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:29:25.933195 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:29:25.946704 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb1eaf9-0b8c-49eb-a9f0-3aeae7344c40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:29:25.947336 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb1eaf9-0b8c-49eb-a9f0-3aeae7344c40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:29:25.947618 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb1eaf9-0b8c-49eb-a9f0-3aeae7344c40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:29:25.947895 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb1eaf9-0b8c-49eb-a9f0-3aeae7344c40 admin admin] PATH_INFO: `/v3/projects/fb863f706c3846db958c9b7fe367c3f5` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:29:25.949093 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb1eaf9-0b8c-49eb-a9f0-3aeae7344c40 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fb863f706c3846db958c9b7fe367c3f5)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:29:25.950990 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb1eaf9-0b8c-49eb-a9f0-3aeae7344c40 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:29:26.062702 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-0cb1eaf9-0b8c-49eb-a9f0-3aeae7344c40 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fb863f706c3846db958c9b7fe367c3f5', 'request_id': 'req-0cb1eaf9-0b8c-49eb-a9f0-3aeae7344c40'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:29:26.068097 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-0cb1eaf9-0b8c-49eb-a9f0-3aeae7344c40 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fb863f706c3846db958c9b7fe367c3f5', 'request_id': 'req-0cb1eaf9-0b8c-49eb-a9f0-3aeae7344c40'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:29:26.136114 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-0cb1eaf9-0b8c-49eb-a9f0-3aeae7344c40 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fb863f706c3846db958c9b7fe367c3f5', 'request_id': 'req-0cb1eaf9-0b8c-49eb-a9f0-3aeae7344c40'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:29:26.148448 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-0cb1eaf9-0b8c-49eb-a9f0-3aeae7344c40 admin admin] The token cache is being invalidate because project fb863f706c3846db958c9b7fe367c3f5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81750) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 01:29:26.148762 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-0cb1eaf9-0b8c-49eb-a9f0-3aeae7344c40 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:29:26.149432 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-0cb1eaf9-0b8c-49eb-a9f0-3aeae7344c40 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:29:26.151277 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1244/2484] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 01:29:25 2022] DELETE /identity/v3/projects/fb863f706c3846db958c9b7fe367c3f5 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 01:29:26.156218 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94332cb-ee04-4c78-9e57-3f2666242a32 None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:29:26.156513 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94332cb-ee04-4c78-9e57-3f2666242a32 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:29:26.156824 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94332cb-ee04-4c78-9e57-3f2666242a32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:29:26.166181 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:29:26.177138 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-c94332cb-ee04-4c78-9e57-3f2666242a32 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:29:26.178363 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-c94332cb-ee04-4c78-9e57-3f2666242a32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:29:26.247435 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1241/2485] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:29:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:29:26.256502 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1db4f9-23ba-4a60-a925-b650cdce39be None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:29:26.256831 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1db4f9-23ba-4a60-a925-b650cdce39be None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:29:26.259152 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1db4f9-23ba-4a60-a925-b650cdce39be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:29:26.278378 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-dc1db4f9-23ba-4a60-a925-b650cdce39be None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:29:26.279687 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-dc1db4f9-23ba-4a60-a925-b650cdce39be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:29:26.288671 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1245/2486] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:29:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:29:26.292857 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94332cb-ee04-4c78-9e57-3f2666242a32 None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:29:26.302540 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94332cb-ee04-4c78-9e57-3f2666242a32 None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:29:26.307346 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38648b92-6d28-4e70-8726-76abb56f9bba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:29:26.307976 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38648b92-6d28-4e70-8726-76abb56f9bba admin admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:29:26.308256 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38648b92-6d28-4e70-8726-76abb56f9bba admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:29:26.308530 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38648b92-6d28-4e70-8726-76abb56f9bba admin admin] PATH_INFO: `/v3/domains` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:29:26.309215 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38648b92-6d28-4e70-8726-76abb56f9bba admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:29:26.309520 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38648b92-6d28-4e70-8726-76abb56f9bba admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:29:26.311114 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38648b92-6d28-4e70-8726-76abb56f9bba admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:29:26.322567 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1242/2487] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Aug 31 01:29:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:29:26.326953 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1db4f9-23ba-4a60-a925-b650cdce39be None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:29:26.337032 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1db4f9-23ba-4a60-a925-b650cdce39be None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:29:26.341805 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6ce374-816c-43b4-bf3d-556fd7eefcd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:29:26.342396 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6ce374-816c-43b4-bf3d-556fd7eefcd5 admin admin] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:29:26.342677 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6ce374-816c-43b4-bf3d-556fd7eefcd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:29:26.342957 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6ce374-816c-43b4-bf3d-556fd7eefcd5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:29:26.343584 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6ce374-816c-43b4-bf3d-556fd7eefcd5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:29:26.345532 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6ce374-816c-43b4-bf3d-556fd7eefcd5 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:29:26.440329 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1246/2488] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 01:29:26 2022] POST /identity/v3/projects => generated 371 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:29:26.444749 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38648b92-6d28-4e70-8726-76abb56f9bba admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:29:26.455137 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38648b92-6d28-4e70-8726-76abb56f9bba admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:29:26.511129 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93142b2-1a85-4585-9d9e-bd2dce7741a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:29:26.511664 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93142b2-1a85-4585-9d9e-bd2dce7741a4 admin admin] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:29:26.511949 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93142b2-1a85-4585-9d9e-bd2dce7741a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:29:26.512228 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93142b2-1a85-4585-9d9e-bd2dce7741a4 admin admin] PATH_INFO: `/v3/users` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:29:26.512918 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93142b2-1a85-4585-9d9e-bd2dce7741a4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:29:26.514851 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93142b2-1a85-4585-9d9e-bd2dce7741a4 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:29:26.588197 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1243/2489] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 01:29:26 2022] POST /identity/v3/users => generated 286 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:29:26.595972 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6ce374-816c-43b4-bf3d-556fd7eefcd5 admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:29:26.606279 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6ce374-816c-43b4-bf3d-556fd7eefcd5 admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:29:26.611066 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b95cf7-d572-44a8-8d54-c0fb150a40dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:29:26.611720 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b95cf7-d572-44a8-8d54-c0fb150a40dc admin admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:29:26.616934 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b95cf7-d572-44a8-8d54-c0fb150a40dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:29:26.616934 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b95cf7-d572-44a8-8d54-c0fb150a40dc admin admin] PATH_INFO: `/v3/roles` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:29:26.616934 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b95cf7-d572-44a8-8d54-c0fb150a40dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:29:26.616934 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b95cf7-d572-44a8-8d54-c0fb150a40dc admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:29:26.624511 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1247/2490] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:29:26 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:29:26.628345 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93142b2-1a85-4585-9d9e-bd2dce7741a4 admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:29:26.639326 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93142b2-1a85-4585-9d9e-bd2dce7741a4 admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:29:26.644232 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e879b97-35c7-4d38-b711-9e0112e3fd9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:29:26.644863 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e879b97-35c7-4d38-b711-9e0112e3fd9a admin admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:29:26.645172 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e879b97-35c7-4d38-b711-9e0112e3fd9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:29:26.645453 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e879b97-35c7-4d38-b711-9e0112e3fd9a admin admin] PATH_INFO: `/v3/roles` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:29:26.646164 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e879b97-35c7-4d38-b711-9e0112e3fd9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:29:26.648207 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e879b97-35c7-4d38-b711-9e0112e3fd9a admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:29:26.659433 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1244/2491] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:29:26 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:29:26.663454 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b95cf7-d572-44a8-8d54-c0fb150a40dc admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:29:26.673429 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b95cf7-d572-44a8-8d54-c0fb150a40dc admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:29:26.678054 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bb58a6-1a5b-43c8-913e-cb5dba61a637 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:29:26.678689 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bb58a6-1a5b-43c8-913e-cb5dba61a637 admin admin] REQUEST_METHOD: `PUT` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:29:26.678967 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bb58a6-1a5b-43c8-913e-cb5dba61a637 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:29:26.679241 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bb58a6-1a5b-43c8-913e-cb5dba61a637 admin admin] PATH_INFO: `/v3/projects/1572b045b5614756849f3f7212b688f7/users/388a8b225f094fceb021e6db0cd88a41/roles/d0b9768f4d014766b9326387f2e11f34` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:29:26.694755 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bb58a6-1a5b-43c8-913e-cb5dba61a637 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1572b045b5614756849f3f7212b688f7, user_id=388a8b225f094fceb021e6db0cd88a41, role_id=d0b9768f4d014766b9326387f2e11f34)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:29:26.697509 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bb58a6-1a5b-43c8-913e-cb5dba61a637 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:29:26.793441 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1248/2492] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 01:29:26 2022] PUT /identity/v3/projects/1572b045b5614756849f3f7212b688f7/users/388a8b225f094fceb021e6db0cd88a41/roles/d0b9768f4d014766b9326387f2e11f34 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:29:26.803383 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15a2ebe-a381-4ab7-8dfc-00474f785dd5 None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:29:26.808688 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15a2ebe-a381-4ab7-8dfc-00474f785dd5 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:29:26.808688 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15a2ebe-a381-4ab7-8dfc-00474f785dd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:29:26.831614 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-e15a2ebe-a381-4ab7-8dfc-00474f785dd5 None None] MFA Rules not processed for user `388a8b225f094fceb021e6db0cd88a41`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:29:26.832856 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-e15a2ebe-a381-4ab7-8dfc-00474f785dd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:29:26.893809 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:29:26.897714 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1245/2493] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:29:26 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:29:26.909786 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bb58a6-1a5b-43c8-913e-cb5dba61a637 admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:29:26.911484 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-b1bb58a6-1a5b-43c8-913e-cb5dba61a637 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:29:26.985501 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bb58a6-1a5b-43c8-913e-cb5dba61a637 admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:29:26.991699 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1bb438-c6af-4be6-8dfd-bc2866483f41 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8744c9de803048f583d0317d01f6a540', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:29:26.992336 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1bb438-c6af-4be6-8dfd-bc2866483f41 service nova] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:29:26.992621 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1bb438-c6af-4be6-8dfd-bc2866483f41 service nova] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:29:26.992946 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1bb438-c6af-4be6-8dfd-bc2866483f41 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:29:27.000642 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1bb438-c6af-4be6-8dfd-bc2866483f41 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:29:27.002254 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1bb438-c6af-4be6-8dfd-bc2866483f41 service nova] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:29:27.005784 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:29:27.011894 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1249/2494] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:29:26 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:29:31.302249 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15a2ebe-a381-4ab7-8dfc-00474f785dd5 None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:29:31.366968 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15a2ebe-a381-4ab7-8dfc-00474f785dd5 None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:29:31.371663 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2097cf-5866-41d2-95ae-16fa66c34401 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:29:31.372305 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2097cf-5866-41d2-95ae-16fa66c34401 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:29:31.372589 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2097cf-5866-41d2-95ae-16fa66c34401 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:29:31.372919 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2097cf-5866-41d2-95ae-16fa66c34401 admin admin] PATH_INFO: `/v3/users/388a8b225f094fceb021e6db0cd88a41` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:29:31.374045 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2097cf-5866-41d2-95ae-16fa66c34401 admin admin] RBAC: Authorizing `identity:delete_user(user_id=388a8b225f094fceb021e6db0cd88a41)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:29:31.376007 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2097cf-5866-41d2-95ae-16fa66c34401 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:29:31.538272 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-0d2097cf-5866-41d2-95ae-16fa66c34401 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '388a8b225f094fceb021e6db0cd88a41'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:29:31.627288 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-0d2097cf-5866-41d2-95ae-16fa66c34401 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '388a8b225f094fceb021e6db0cd88a41'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:29:31.627909 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-0d2097cf-5866-41d2-95ae-16fa66c34401 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '388a8b225f094fceb021e6db0cd88a41'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:29:31.635740 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-0d2097cf-5866-41d2-95ae-16fa66c34401 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '388a8b225f094fceb021e6db0cd88a41'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:29:31.636437 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-0d2097cf-5866-41d2-95ae-16fa66c34401 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '388a8b225f094fceb021e6db0cd88a41'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:29:31.654686 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1246/2495] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 01:29:31 2022] DELETE /identity/v3/users/388a8b225f094fceb021e6db0cd88a41 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:29:31.659219 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9672b4-099f-4b9e-a39b-60595a977b7a None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:29:31.659705 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9672b4-099f-4b9e-a39b-60595a977b7a None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:29:31.659803 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9672b4-099f-4b9e-a39b-60595a977b7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:29:31.679040 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-7a9672b4-099f-4b9e-a39b-60595a977b7a None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:29:31.680252 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-7a9672b4-099f-4b9e-a39b-60595a977b7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:29:31.741336 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1250/2496] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:29:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:29:31.754170 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2097cf-5866-41d2-95ae-16fa66c34401 admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:29:31.755632 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-0d2097cf-5866-41d2-95ae-16fa66c34401 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:29:31.816404 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2097cf-5866-41d2-95ae-16fa66c34401 admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:29:31.820157 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae9b280-349d-4403-832b-04c5b6ad348f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0817c3320d5d41d8bba0550c8ae8dd60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:29:31.820793 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae9b280-349d-4403-832b-04c5b6ad348f service neutron] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:29:31.821081 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae9b280-349d-4403-832b-04c5b6ad348f service neutron] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:29:31.821364 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae9b280-349d-4403-832b-04c5b6ad348f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:29:31.831608 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae9b280-349d-4403-832b-04c5b6ad348f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:29:31.833667 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae9b280-349d-4403-832b-04c5b6ad348f service neutron] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:29:31.844433 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1247/2497] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:29:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:29:32.503608 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9672b4-099f-4b9e-a39b-60595a977b7a None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:29:32.503608 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-7a9672b4-099f-4b9e-a39b-60595a977b7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:29:32.545723 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9672b4-099f-4b9e-a39b-60595a977b7a None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:29:32.550819 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7b4c12-92c8-4075-aac1-e33d659b0da0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:29:32.551471 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7b4c12-92c8-4075-aac1-e33d659b0da0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:29:32.551758 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7b4c12-92c8-4075-aac1-e33d659b0da0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:29:32.552807 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7b4c12-92c8-4075-aac1-e33d659b0da0 admin admin] PATH_INFO: `/v3/projects/1572b045b5614756849f3f7212b688f7` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:29:32.553800 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7b4c12-92c8-4075-aac1-e33d659b0da0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1572b045b5614756849f3f7212b688f7)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:29:32.555781 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7b4c12-92c8-4075-aac1-e33d659b0da0 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:29:32.714022 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-9d7b4c12-92c8-4075-aac1-e33d659b0da0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1572b045b5614756849f3f7212b688f7', 'request_id': 'req-9d7b4c12-92c8-4075-aac1-e33d659b0da0'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:29:32.720440 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-9d7b4c12-92c8-4075-aac1-e33d659b0da0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1572b045b5614756849f3f7212b688f7', 'request_id': 'req-9d7b4c12-92c8-4075-aac1-e33d659b0da0'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:29:32.857225 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-9d7b4c12-92c8-4075-aac1-e33d659b0da0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1572b045b5614756849f3f7212b688f7', 'request_id': 'req-9d7b4c12-92c8-4075-aac1-e33d659b0da0'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:29:32.870332 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-9d7b4c12-92c8-4075-aac1-e33d659b0da0 admin admin] The token cache is being invalidate because project 1572b045b5614756849f3f7212b688f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81750) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 01:29:32.870636 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-9d7b4c12-92c8-4075-aac1-e33d659b0da0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:29:32.871315 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-9d7b4c12-92c8-4075-aac1-e33d659b0da0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:29:32.878416 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c91123-f8e2-4f56-abd8-d31f554c2787 None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:29:32.878957 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c91123-f8e2-4f56-abd8-d31f554c2787 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:29:32.879043 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1251/2498] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 01:29:32 2022] DELETE /identity/v3/projects/1572b045b5614756849f3f7212b688f7 => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:29:32.879125 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c91123-f8e2-4f56-abd8-d31f554c2787 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:29:32.909910 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-d2c91123-f8e2-4f56-abd8-d31f554c2787 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:29:32.911133 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-d2c91123-f8e2-4f56-abd8-d31f554c2787 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:29:32.931379 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:29:32.973733 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1248/2499] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:29:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:29:32.981589 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd720a27-b7eb-400a-8661-90fec55d3be5 None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:29:32.981884 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd720a27-b7eb-400a-8661-90fec55d3be5 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:29:32.988552 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd720a27-b7eb-400a-8661-90fec55d3be5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:29:33.002366 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-fd720a27-b7eb-400a-8661-90fec55d3be5 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:29:33.003524 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-fd720a27-b7eb-400a-8661-90fec55d3be5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:29:33.012777 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1252/2500] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:29:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:29:33.017311 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c91123-f8e2-4f56-abd8-d31f554c2787 None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:29:33.027415 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c91123-f8e2-4f56-abd8-d31f554c2787 None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:29:33.032226 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bae3ab9-8e25-41aa-91c8-2477d680a031 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:29:33.032875 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bae3ab9-8e25-41aa-91c8-2477d680a031 admin admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:29:33.033163 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bae3ab9-8e25-41aa-91c8-2477d680a031 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:29:33.033442 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bae3ab9-8e25-41aa-91c8-2477d680a031 admin admin] PATH_INFO: `/v3/domains` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:29:33.034093 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bae3ab9-8e25-41aa-91c8-2477d680a031 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:29:33.034399 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bae3ab9-8e25-41aa-91c8-2477d680a031 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:29:33.036006 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bae3ab9-8e25-41aa-91c8-2477d680a031 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:29:33.052252 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1249/2501] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Aug 31 01:29:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:29:33.054055 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd720a27-b7eb-400a-8661-90fec55d3be5 None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:29:33.063529 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd720a27-b7eb-400a-8661-90fec55d3be5 None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:29:33.069272 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e386771-625e-4b89-bea4-96768b79b753 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:29:33.069959 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e386771-625e-4b89-bea4-96768b79b753 admin admin] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:29:33.070244 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e386771-625e-4b89-bea4-96768b79b753 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:29:33.070523 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e386771-625e-4b89-bea4-96768b79b753 admin admin] PATH_INFO: `/v3/projects` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:29:33.071156 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e386771-625e-4b89-bea4-96768b79b753 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:29:33.073203 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e386771-625e-4b89-bea4-96768b79b753 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:29:33.134345 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1253/2502] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 01:29:33 2022] POST /identity/v3/projects => generated 373 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:29:33.138670 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bae3ab9-8e25-41aa-91c8-2477d680a031 admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:29:33.148998 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bae3ab9-8e25-41aa-91c8-2477d680a031 admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:29:33.207901 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3734470-e520-4f7e-b321-2a9dccb52eff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:29:33.208508 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3734470-e520-4f7e-b321-2a9dccb52eff admin admin] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:29:33.208876 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3734470-e520-4f7e-b321-2a9dccb52eff admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:29:33.209178 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3734470-e520-4f7e-b321-2a9dccb52eff admin admin] PATH_INFO: `/v3/users` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:29:33.209806 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3734470-e520-4f7e-b321-2a9dccb52eff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:29:33.211780 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3734470-e520-4f7e-b321-2a9dccb52eff admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:29:33.294403 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1250/2503] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 01:29:33 2022] POST /identity/v3/users => generated 287 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:29:33.295698 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e386771-625e-4b89-bea4-96768b79b753 admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:29:33.307911 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e386771-625e-4b89-bea4-96768b79b753 admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:29:33.312663 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082d8f43-8fae-4268-81a2-e384b6cd91b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:29:33.313296 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082d8f43-8fae-4268-81a2-e384b6cd91b8 admin admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:29:33.313719 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082d8f43-8fae-4268-81a2-e384b6cd91b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:29:33.313994 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082d8f43-8fae-4268-81a2-e384b6cd91b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:29:33.314650 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082d8f43-8fae-4268-81a2-e384b6cd91b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:29:33.316236 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082d8f43-8fae-4268-81a2-e384b6cd91b8 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:29:33.326747 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1254/2504] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:29:33 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:29:33.332669 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3734470-e520-4f7e-b321-2a9dccb52eff admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:29:33.343978 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3734470-e520-4f7e-b321-2a9dccb52eff admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:29:33.349122 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da51c6e0-bc1a-47d2-bbaa-80e30a990092 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:29:33.350292 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da51c6e0-bc1a-47d2-bbaa-80e30a990092 admin admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:29:33.350582 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da51c6e0-bc1a-47d2-bbaa-80e30a990092 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:29:33.355298 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da51c6e0-bc1a-47d2-bbaa-80e30a990092 admin admin] PATH_INFO: `/v3/roles` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:29:33.356042 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da51c6e0-bc1a-47d2-bbaa-80e30a990092 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:29:33.357724 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da51c6e0-bc1a-47d2-bbaa-80e30a990092 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:29:33.380328 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1251/2505] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:29:33 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:29:33.381850 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082d8f43-8fae-4268-81a2-e384b6cd91b8 admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:29:33.392590 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082d8f43-8fae-4268-81a2-e384b6cd91b8 admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:29:33.397511 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13800e46-207c-4844-ba3b-c49c4150f129 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:29:33.398480 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13800e46-207c-4844-ba3b-c49c4150f129 admin admin] REQUEST_METHOD: `PUT` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:29:33.398789 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13800e46-207c-4844-ba3b-c49c4150f129 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:29:33.399114 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13800e46-207c-4844-ba3b-c49c4150f129 admin admin] PATH_INFO: `/v3/projects/11fd3ccf6b67442a94afe88f399863aa/users/7e18507e425e4d3bbcfb6fb0c34e086e/roles/d0b9768f4d014766b9326387f2e11f34` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:29:33.416561 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13800e46-207c-4844-ba3b-c49c4150f129 admin admin] RBAC: Authorizing `identity:create_grant(project_id=11fd3ccf6b67442a94afe88f399863aa, user_id=7e18507e425e4d3bbcfb6fb0c34e086e, role_id=d0b9768f4d014766b9326387f2e11f34)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:29:33.419367 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13800e46-207c-4844-ba3b-c49c4150f129 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:29:33.508317 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1255/2506] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 01:29:33 2022] PUT /identity/v3/projects/11fd3ccf6b67442a94afe88f399863aa/users/7e18507e425e4d3bbcfb6fb0c34e086e/roles/d0b9768f4d014766b9326387f2e11f34 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 01:29:33.520261 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3fb48c-a304-4940-adda-da8c20466223 None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:29:33.520577 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3fb48c-a304-4940-adda-da8c20466223 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:29:33.520904 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3fb48c-a304-4940-adda-da8c20466223 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:29:33.551417 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-8b3fb48c-a304-4940-adda-da8c20466223 None None] MFA Rules not processed for user `7e18507e425e4d3bbcfb6fb0c34e086e`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:29:33.552830 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-8b3fb48c-a304-4940-adda-da8c20466223 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:29:33.630238 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1252/2507] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:29:33 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:29:33.634527 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13800e46-207c-4844-ba3b-c49c4150f129 admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:29:33.658316 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:29:33.713355 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13800e46-207c-4844-ba3b-c49c4150f129 admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:29:33.718463 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60ea402-fee7-4fab-a8b6-5ea53b96dce3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:29:33.719299 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60ea402-fee7-4fab-a8b6-5ea53b96dce3 admin admin] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:29:33.719705 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60ea402-fee7-4fab-a8b6-5ea53b96dce3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:29:33.719992 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60ea402-fee7-4fab-a8b6-5ea53b96dce3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:29:33.720629 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60ea402-fee7-4fab-a8b6-5ea53b96dce3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:29:33.722539 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60ea402-fee7-4fab-a8b6-5ea53b96dce3 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:29:33.836011 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1256/2508] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 01:29:33 2022] POST /identity/v3/projects => generated 373 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:29:33.839879 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3fb48c-a304-4940-adda-da8c20466223 None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:29:33.851390 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3fb48c-a304-4940-adda-da8c20466223 None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:29:33.864161 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:29:33.910257 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c0b0d8-0b8a-4b40-b870-c865a5f01f0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:29:33.910818 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c0b0d8-0b8a-4b40-b870-c865a5f01f0e admin admin] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:29:33.911105 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c0b0d8-0b8a-4b40-b870-c865a5f01f0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:29:33.911382 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c0b0d8-0b8a-4b40-b870-c865a5f01f0e admin admin] PATH_INFO: `/v3/users` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:29:33.912032 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c0b0d8-0b8a-4b40-b870-c865a5f01f0e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:29:33.914544 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c0b0d8-0b8a-4b40-b870-c865a5f01f0e admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:29:34.005094 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1253/2509] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 01:29:33 2022] POST /identity/v3/users => generated 293 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:29:34.009796 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60ea402-fee7-4fab-a8b6-5ea53b96dce3 admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:29:34.020017 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60ea402-fee7-4fab-a8b6-5ea53b96dce3 admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:29:34.025372 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ef18d9-6c8b-46e4-bbac-d181c3b253e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:29:34.025970 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ef18d9-6c8b-46e4-bbac-d181c3b253e8 admin admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:29:34.026341 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ef18d9-6c8b-46e4-bbac-d181c3b253e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:29:34.026721 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ef18d9-6c8b-46e4-bbac-d181c3b253e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:29:34.027364 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ef18d9-6c8b-46e4-bbac-d181c3b253e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:29:34.028998 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ef18d9-6c8b-46e4-bbac-d181c3b253e8 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:29:34.039322 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1257/2510] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:29:34 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:29:34.043351 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c0b0d8-0b8a-4b40-b870-c865a5f01f0e admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:29:34.053051 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c0b0d8-0b8a-4b40-b870-c865a5f01f0e admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:29:34.057559 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ef7004-44af-4494-a4f3-cc7072b7134e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:29:34.058192 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ef7004-44af-4494-a4f3-cc7072b7134e admin admin] REQUEST_METHOD: `PUT` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:29:34.058473 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ef7004-44af-4494-a4f3-cc7072b7134e admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:29:34.058748 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ef7004-44af-4494-a4f3-cc7072b7134e admin admin] PATH_INFO: `/v3/projects/2b562496d2a04f419921602e5a515919/users/22287b3e49ed4b00a830b2449b03638d/roles/d009411496b8452b88e77ffb8db015a7` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:29:34.073334 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ef7004-44af-4494-a4f3-cc7072b7134e admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b562496d2a04f419921602e5a515919, user_id=22287b3e49ed4b00a830b2449b03638d, role_id=d009411496b8452b88e77ffb8db015a7)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:29:34.075972 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ef7004-44af-4494-a4f3-cc7072b7134e admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:29:34.153732 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1254/2511] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 01:29:34 2022] PUT /identity/v3/projects/2b562496d2a04f419921602e5a515919/users/22287b3e49ed4b00a830b2449b03638d/roles/d009411496b8452b88e77ffb8db015a7 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:29:34.157644 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ef18d9-6c8b-46e4-bbac-d181c3b253e8 admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:29:34.167914 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ef18d9-6c8b-46e4-bbac-d181c3b253e8 admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:29:34.223990 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd4e686-1226-4c53-b288-f9624467fcbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:29:34.224665 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd4e686-1226-4c53-b288-f9624467fcbf admin admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:29:34.224981 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd4e686-1226-4c53-b288-f9624467fcbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:29:34.225261 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd4e686-1226-4c53-b288-f9624467fcbf admin admin] PATH_INFO: `/v3/roles` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:29:34.225945 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd4e686-1226-4c53-b288-f9624467fcbf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:29:34.227550 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd4e686-1226-4c53-b288-f9624467fcbf admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:29:34.237314 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1258/2512] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:29:34 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 01:29:34.241534 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ef7004-44af-4494-a4f3-cc7072b7134e admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:29:34.251899 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ef7004-44af-4494-a4f3-cc7072b7134e admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:29:34.257059 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747fc5c4-7b88-4a32-a1d2-ba9f7c21aa82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:29:34.257714 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747fc5c4-7b88-4a32-a1d2-ba9f7c21aa82 admin admin] REQUEST_METHOD: `PUT` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:29:34.257995 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747fc5c4-7b88-4a32-a1d2-ba9f7c21aa82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:29:34.258271 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747fc5c4-7b88-4a32-a1d2-ba9f7c21aa82 admin admin] PATH_INFO: `/v3/projects/2b562496d2a04f419921602e5a515919/users/22287b3e49ed4b00a830b2449b03638d/roles/d009411496b8452b88e77ffb8db015a7` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:29:34.260191 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747fc5c4-7b88-4a32-a1d2-ba9f7c21aa82 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b562496d2a04f419921602e5a515919, user_id=22287b3e49ed4b00a830b2449b03638d, role_id=d009411496b8452b88e77ffb8db015a7)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:29:34.262864 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747fc5c4-7b88-4a32-a1d2-ba9f7c21aa82 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:29:34.273627 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1255/2513] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 01:29:34 2022] PUT /identity/v3/projects/2b562496d2a04f419921602e5a515919/users/22287b3e49ed4b00a830b2449b03638d/roles/d009411496b8452b88e77ffb8db015a7 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:29:34.283649 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4141ca6e-4100-425d-9a35-c13588f8fbc5 None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:29:34.283950 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4141ca6e-4100-425d-9a35-c13588f8fbc5 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:29:34.284234 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4141ca6e-4100-425d-9a35-c13588f8fbc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:29:34.311200 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-4141ca6e-4100-425d-9a35-c13588f8fbc5 None None] MFA Rules not processed for user `22287b3e49ed4b00a830b2449b03638d`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:29:34.312422 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-4141ca6e-4100-425d-9a35-c13588f8fbc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:29:34.385675 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1259/2514] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:29:34 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:29:34.393153 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747fc5c4-7b88-4a32-a1d2-ba9f7c21aa82 admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:29:34.395948 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-747fc5c4-7b88-4a32-a1d2-ba9f7c21aa82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:29:34.453833 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747fc5c4-7b88-4a32-a1d2-ba9f7c21aa82 admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:29:34.457776 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20450ec-136c-45d6-a296-416b22d2fc23 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5c8fc69c518412ba1aad61a9f4f558c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:29:34.458357 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20450ec-136c-45d6-a296-416b22d2fc23 service cinder] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:29:34.458639 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20450ec-136c-45d6-a296-416b22d2fc23 service cinder] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:29:34.458913 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20450ec-136c-45d6-a296-416b22d2fc23 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:29:34.467943 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20450ec-136c-45d6-a296-416b22d2fc23 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:29:34.469494 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20450ec-136c-45d6-a296-416b22d2fc23 service cinder] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:29:34.479037 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1256/2515] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:29:34 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:29:34.675547 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4141ca6e-4100-425d-9a35-c13588f8fbc5 None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:29:34.685557 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4141ca6e-4100-425d-9a35-c13588f8fbc5 None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:29:34.689098 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd48be2e-231f-4e54-b345-ce339ed69b82 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5c8fc69c518412ba1aad61a9f4f558c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:29:34.689697 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd48be2e-231f-4e54-b345-ce339ed69b82 service cinder] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:29:34.689979 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd48be2e-231f-4e54-b345-ce339ed69b82 service cinder] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:29:34.690254 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd48be2e-231f-4e54-b345-ce339ed69b82 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:29:34.700006 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:29:34.746829 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd48be2e-231f-4e54-b345-ce339ed69b82 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:29:34.748289 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd48be2e-231f-4e54-b345-ce339ed69b82 service cinder] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:29:34.758872 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1260/2516] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:29:34 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:29:44.474774 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=62,keystone:INSERT=5 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:29:44.754644 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=22 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:29:46.460205 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20450ec-136c-45d6-a296-416b22d2fc23 service cinder] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:29:46.461655 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-a20450ec-136c-45d6-a296-416b22d2fc23 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:29:46.521630 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20450ec-136c-45d6-a296-416b22d2fc23 service cinder] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:29:46.525302 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f99c4b-699c-4a75-850c-16f30d9fc9a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0817c3320d5d41d8bba0550c8ae8dd60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:29:46.526516 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f99c4b-699c-4a75-850c-16f30d9fc9a5 service neutron] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:29:46.526836 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f99c4b-699c-4a75-850c-16f30d9fc9a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:29:46.527112 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f99c4b-699c-4a75-850c-16f30d9fc9a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:29:46.528920 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-b5f99c4b-699c-4a75-850c-16f30d9fc9a5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:29:46.600026 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f99c4b-699c-4a75-850c-16f30d9fc9a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:29:46.601510 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f99c4b-699c-4a75-850c-16f30d9fc9a5 service neutron] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:29:46.611137 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1257/2517] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:29:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:29:50.453119 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd48be2e-231f-4e54-b345-ce339ed69b82 service cinder] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:29:50.454885 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-cd48be2e-231f-4e54-b345-ce339ed69b82 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:29:50.470079 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd48be2e-231f-4e54-b345-ce339ed69b82 service cinder] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:29:50.475957 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175f3f4c-b555-41a9-b986-9687a782103b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:29:50.476695 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175f3f4c-b555-41a9-b986-9687a782103b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:29:50.480064 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175f3f4c-b555-41a9-b986-9687a782103b admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:29:50.480064 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175f3f4c-b555-41a9-b986-9687a782103b admin admin] PATH_INFO: `/v3/users/efaf57b6482149ca86a9bba26d7654f2` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:29:50.480064 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175f3f4c-b555-41a9-b986-9687a782103b admin admin] RBAC: Authorizing `identity:delete_user(user_id=efaf57b6482149ca86a9bba26d7654f2)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:29:50.481086 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175f3f4c-b555-41a9-b986-9687a782103b admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:29:50.686657 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-175f3f4c-b555-41a9-b986-9687a782103b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'efaf57b6482149ca86a9bba26d7654f2'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:29:50.701685 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-175f3f4c-b555-41a9-b986-9687a782103b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'efaf57b6482149ca86a9bba26d7654f2'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:29:50.702574 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-175f3f4c-b555-41a9-b986-9687a782103b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'efaf57b6482149ca86a9bba26d7654f2'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:29:50.711308 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-175f3f4c-b555-41a9-b986-9687a782103b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'efaf57b6482149ca86a9bba26d7654f2'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:29:50.787600 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-175f3f4c-b555-41a9-b986-9687a782103b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'efaf57b6482149ca86a9bba26d7654f2'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:29:50.794494 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1261/2518] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 01:29:50 2022] DELETE /identity/v3/users/efaf57b6482149ca86a9bba26d7654f2 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:29:51.097860 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f99c4b-699c-4a75-850c-16f30d9fc9a5 service neutron] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:29:51.108078 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f99c4b-699c-4a75-850c-16f30d9fc9a5 service neutron] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:29:51.167935 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b3d58d-541b-46ed-bcee-8f83c653fff8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:29:51.168572 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b3d58d-541b-46ed-bcee-8f83c653fff8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:29:51.168898 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b3d58d-541b-46ed-bcee-8f83c653fff8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:29:51.169178 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b3d58d-541b-46ed-bcee-8f83c653fff8 admin admin] PATH_INFO: `/v3/projects/53d2295ada114ea3b6b5001c4c197f35` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:29:51.170330 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b3d58d-541b-46ed-bcee-8f83c653fff8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=53d2295ada114ea3b6b5001c4c197f35)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:29:51.172251 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b3d58d-541b-46ed-bcee-8f83c653fff8 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:29:51.332104 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-64b3d58d-541b-46ed-bcee-8f83c653fff8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '53d2295ada114ea3b6b5001c4c197f35', 'request_id': 'req-64b3d58d-541b-46ed-bcee-8f83c653fff8'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:29:51.334038 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:29:51.337962 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-64b3d58d-541b-46ed-bcee-8f83c653fff8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '53d2295ada114ea3b6b5001c4c197f35', 'request_id': 'req-64b3d58d-541b-46ed-bcee-8f83c653fff8'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:29:51.344227 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-64b3d58d-541b-46ed-bcee-8f83c653fff8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '53d2295ada114ea3b6b5001c4c197f35', 'request_id': 'req-64b3d58d-541b-46ed-bcee-8f83c653fff8'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:29:51.409740 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-64b3d58d-541b-46ed-bcee-8f83c653fff8 admin admin] The token cache is being invalidate because project 53d2295ada114ea3b6b5001c4c197f35 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81749) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 01:29:51.410029 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-64b3d58d-541b-46ed-bcee-8f83c653fff8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:29:51.410637 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-64b3d58d-541b-46ed-bcee-8f83c653fff8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:29:51.412098 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1258/2519] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 01:29:51 2022] DELETE /identity/v3/projects/53d2295ada114ea3b6b5001c4c197f35 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:29:51.416999 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdda33bf-374b-4f93-836e-dd50d7a52be6 None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:29:51.417294 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdda33bf-374b-4f93-836e-dd50d7a52be6 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:29:51.417574 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdda33bf-374b-4f93-836e-dd50d7a52be6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:29:51.437669 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-fdda33bf-374b-4f93-836e-dd50d7a52be6 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:29:51.438870 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-fdda33bf-374b-4f93-836e-dd50d7a52be6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:29:51.498684 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1262/2520] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:29:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:29:51.506758 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a75f5fa-32d9-4081-b989-e872ccb5ab07 None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:29:51.507049 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a75f5fa-32d9-4081-b989-e872ccb5ab07 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:29:51.507323 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a75f5fa-32d9-4081-b989-e872ccb5ab07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:29:51.529174 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-4a75f5fa-32d9-4081-b989-e872ccb5ab07 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:29:51.530457 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-4a75f5fa-32d9-4081-b989-e872ccb5ab07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:29:51.539046 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1259/2521] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:29:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:29:51.544006 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdda33bf-374b-4f93-836e-dd50d7a52be6 None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:29:51.553889 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdda33bf-374b-4f93-836e-dd50d7a52be6 None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:29:51.559063 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba1da67-b09b-442e-90ed-b9a61c67a301 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:29:51.559661 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba1da67-b09b-442e-90ed-b9a61c67a301 admin admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:29:51.559937 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba1da67-b09b-442e-90ed-b9a61c67a301 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:29:51.560207 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba1da67-b09b-442e-90ed-b9a61c67a301 admin admin] PATH_INFO: `/v3/domains` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:29:51.560879 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba1da67-b09b-442e-90ed-b9a61c67a301 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:29:51.561184 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba1da67-b09b-442e-90ed-b9a61c67a301 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:29:51.562785 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba1da67-b09b-442e-90ed-b9a61c67a301 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:29:51.578330 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1263/2522] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Aug 31 01:29:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 01:29:51.579445 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a75f5fa-32d9-4081-b989-e872ccb5ab07 None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:29:51.589497 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a75f5fa-32d9-4081-b989-e872ccb5ab07 None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:29:51.594213 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed0c56e-e108-4cf7-b8c5-3149473c6ac8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:29:51.594956 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed0c56e-e108-4cf7-b8c5-3149473c6ac8 admin admin] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:29:51.595350 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed0c56e-e108-4cf7-b8c5-3149473c6ac8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:29:51.596500 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed0c56e-e108-4cf7-b8c5-3149473c6ac8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:29:51.597198 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed0c56e-e108-4cf7-b8c5-3149473c6ac8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:29:51.599377 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed0c56e-e108-4cf7-b8c5-3149473c6ac8 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:29:51.676220 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1260/2523] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 01:29:51 2022] POST /identity/v3/projects => generated 403 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:29:51.680297 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba1da67-b09b-442e-90ed-b9a61c67a301 admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:29:51.691487 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba1da67-b09b-442e-90ed-b9a61c67a301 admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:29:51.750844 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865f0220-a068-4f2f-9d92-9c89ac2cc851 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:29:51.751490 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865f0220-a068-4f2f-9d92-9c89ac2cc851 admin admin] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:29:51.751774 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865f0220-a068-4f2f-9d92-9c89ac2cc851 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:29:51.752049 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865f0220-a068-4f2f-9d92-9c89ac2cc851 admin admin] PATH_INFO: `/v3/users` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:29:51.752674 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865f0220-a068-4f2f-9d92-9c89ac2cc851 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:29:51.754618 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865f0220-a068-4f2f-9d92-9c89ac2cc851 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:29:51.763058 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:29:51.844325 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1264/2524] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 01:29:51 2022] POST /identity/v3/users => generated 302 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:29:51.848243 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed0c56e-e108-4cf7-b8c5-3149473c6ac8 admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:29:51.858935 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed0c56e-e108-4cf7-b8c5-3149473c6ac8 admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:29:51.863564 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3c76b7-ca84-476d-86e5-9e1947fff82e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:29:51.864162 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3c76b7-ca84-476d-86e5-9e1947fff82e admin admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:29:51.864445 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3c76b7-ca84-476d-86e5-9e1947fff82e admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:29:51.864746 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3c76b7-ca84-476d-86e5-9e1947fff82e admin admin] PATH_INFO: `/v3/roles` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:29:51.865384 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3c76b7-ca84-476d-86e5-9e1947fff82e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:29:51.870319 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3c76b7-ca84-476d-86e5-9e1947fff82e admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:29:51.877894 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1261/2525] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:29:51 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:29:51.881725 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865f0220-a068-4f2f-9d92-9c89ac2cc851 admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:29:51.892192 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865f0220-a068-4f2f-9d92-9c89ac2cc851 admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:29:51.899757 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913acc3b-b440-46c5-a53c-f9ca77fb73ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:29:51.900387 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913acc3b-b440-46c5-a53c-f9ca77fb73ef admin admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:29:51.900663 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913acc3b-b440-46c5-a53c-f9ca77fb73ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:29:51.900964 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913acc3b-b440-46c5-a53c-f9ca77fb73ef admin admin] PATH_INFO: `/v3/roles` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:29:51.901594 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913acc3b-b440-46c5-a53c-f9ca77fb73ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:29:51.903184 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913acc3b-b440-46c5-a53c-f9ca77fb73ef admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:29:51.913786 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1265/2526] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:29:51 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:29:51.918440 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3c76b7-ca84-476d-86e5-9e1947fff82e admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:29:51.928389 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3c76b7-ca84-476d-86e5-9e1947fff82e admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:29:51.933635 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abbc658-d663-49a7-b444-8870a23db063 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:29:51.934275 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abbc658-d663-49a7-b444-8870a23db063 admin admin] REQUEST_METHOD: `PUT` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:29:51.934560 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abbc658-d663-49a7-b444-8870a23db063 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:29:51.934840 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abbc658-d663-49a7-b444-8870a23db063 admin admin] PATH_INFO: `/v3/projects/aba5b6a5a4374067a75e5c4c4556072e/users/2a955d28f42345e1917f3c21d1f1ec53/roles/d0b9768f4d014766b9326387f2e11f34` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:29:51.948794 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abbc658-d663-49a7-b444-8870a23db063 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aba5b6a5a4374067a75e5c4c4556072e, user_id=2a955d28f42345e1917f3c21d1f1ec53, role_id=d0b9768f4d014766b9326387f2e11f34)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:29:51.951447 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abbc658-d663-49a7-b444-8870a23db063 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:29:52.020766 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1262/2527] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 01:29:51 2022] PUT /identity/v3/projects/aba5b6a5a4374067a75e5c4c4556072e/users/2a955d28f42345e1917f3c21d1f1ec53/roles/d0b9768f4d014766b9326387f2e11f34 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:29:52.031955 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbe5417-84bf-4679-9620-cbae87e4bc27 None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:29:52.032234 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbe5417-84bf-4679-9620-cbae87e4bc27 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:29:52.032514 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbe5417-84bf-4679-9620-cbae87e4bc27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:29:52.063231 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-dbbe5417-84bf-4679-9620-cbae87e4bc27 None None] MFA Rules not processed for user `2a955d28f42345e1917f3c21d1f1ec53`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:29:52.064525 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-dbbe5417-84bf-4679-9620-cbae87e4bc27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:29:52.129377 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1266/2528] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:29:52 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:29:52.133803 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abbc658-d663-49a7-b444-8870a23db063 admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:29:52.196953 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abbc658-d663-49a7-b444-8870a23db063 admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:29:52.201841 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5351f80e-2101-4e73-9d12-161b8023e3ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:29:52.202468 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5351f80e-2101-4e73-9d12-161b8023e3ea admin admin] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:29:52.202757 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5351f80e-2101-4e73-9d12-161b8023e3ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:29:52.203068 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5351f80e-2101-4e73-9d12-161b8023e3ea admin admin] PATH_INFO: `/v3/projects` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:29:52.203700 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5351f80e-2101-4e73-9d12-161b8023e3ea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:29:52.205960 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5351f80e-2101-4e73-9d12-161b8023e3ea admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:29:52.278062 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1263/2529] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 01:29:52 2022] POST /identity/v3/projects => generated 403 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 01:29:52.282795 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbe5417-84bf-4679-9620-cbae87e4bc27 None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:29:52.294361 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbe5417-84bf-4679-9620-cbae87e4bc27 None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:29:52.357912 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8ee41c-0aa1-4b4b-b0de-96f61ee6c586 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:29:52.358583 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8ee41c-0aa1-4b4b-b0de-96f61ee6c586 admin admin] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:29:52.358999 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8ee41c-0aa1-4b4b-b0de-96f61ee6c586 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:29:52.359339 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8ee41c-0aa1-4b4b-b0de-96f61ee6c586 admin admin] PATH_INFO: `/v3/users` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:29:52.363162 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8ee41c-0aa1-4b4b-b0de-96f61ee6c586 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:29:52.363162 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8ee41c-0aa1-4b4b-b0de-96f61ee6c586 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:29:52.438957 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1267/2530] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 01:29:52 2022] POST /identity/v3/users => generated 308 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:29:52.443435 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5351f80e-2101-4e73-9d12-161b8023e3ea admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:29:52.454466 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5351f80e-2101-4e73-9d12-161b8023e3ea admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:29:52.459399 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7415d7d6-fd1f-4f3a-b8e9-67395db9fcc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:29:52.459998 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7415d7d6-fd1f-4f3a-b8e9-67395db9fcc7 admin admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:29:52.460295 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7415d7d6-fd1f-4f3a-b8e9-67395db9fcc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:29:52.463626 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7415d7d6-fd1f-4f3a-b8e9-67395db9fcc7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:29:52.463626 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7415d7d6-fd1f-4f3a-b8e9-67395db9fcc7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:29:52.463626 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7415d7d6-fd1f-4f3a-b8e9-67395db9fcc7 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:29:52.477139 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1264/2531] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:29:52 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:29:52.478912 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8ee41c-0aa1-4b4b-b0de-96f61ee6c586 admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:29:52.489254 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8ee41c-0aa1-4b4b-b0de-96f61ee6c586 admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:29:52.493935 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938500b0-9364-416e-8ddf-6ab74bb860a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:29:52.498104 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938500b0-9364-416e-8ddf-6ab74bb860a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:29:52.498104 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938500b0-9364-416e-8ddf-6ab74bb860a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:29:52.498104 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938500b0-9364-416e-8ddf-6ab74bb860a1 admin admin] PATH_INFO: `/v3/projects/98438cec810d42cab9d73e2e3170622b/users/8bf88c3f52964aeab9d3778bcfb6cb6e/roles/d009411496b8452b88e77ffb8db015a7` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:29:52.516416 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938500b0-9364-416e-8ddf-6ab74bb860a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=98438cec810d42cab9d73e2e3170622b, user_id=8bf88c3f52964aeab9d3778bcfb6cb6e, role_id=d009411496b8452b88e77ffb8db015a7)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:29:52.519607 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938500b0-9364-416e-8ddf-6ab74bb860a1 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:29:52.598491 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1268/2532] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 01:29:52 2022] PUT /identity/v3/projects/98438cec810d42cab9d73e2e3170622b/users/8bf88c3f52964aeab9d3778bcfb6cb6e/roles/d009411496b8452b88e77ffb8db015a7 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:29:52.602601 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7415d7d6-fd1f-4f3a-b8e9-67395db9fcc7 admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:29:52.612936 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7415d7d6-fd1f-4f3a-b8e9-67395db9fcc7 admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:29:52.662197 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:29:52.673366 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757d2d72-8c16-42db-8b8b-56266f2ef4ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:29:52.673985 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757d2d72-8c16-42db-8b8b-56266f2ef4ec admin admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:29:52.674374 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757d2d72-8c16-42db-8b8b-56266f2ef4ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:29:52.674528 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757d2d72-8c16-42db-8b8b-56266f2ef4ec admin admin] PATH_INFO: `/v3/roles` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:29:52.675281 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757d2d72-8c16-42db-8b8b-56266f2ef4ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:29:52.676821 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757d2d72-8c16-42db-8b8b-56266f2ef4ec admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:29:52.688176 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1265/2533] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:29:52 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:29:52.695726 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938500b0-9364-416e-8ddf-6ab74bb860a1 admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:29:52.707559 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938500b0-9364-416e-8ddf-6ab74bb860a1 admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:29:52.715377 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f337124-bcd8-41fc-9431-59ba7b864a2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:29:52.715377 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f337124-bcd8-41fc-9431-59ba7b864a2c admin admin] REQUEST_METHOD: `PUT` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:29:52.715377 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f337124-bcd8-41fc-9431-59ba7b864a2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:29:52.715377 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f337124-bcd8-41fc-9431-59ba7b864a2c admin admin] PATH_INFO: `/v3/projects/98438cec810d42cab9d73e2e3170622b/users/8bf88c3f52964aeab9d3778bcfb6cb6e/roles/d009411496b8452b88e77ffb8db015a7` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:29:52.716427 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f337124-bcd8-41fc-9431-59ba7b864a2c admin admin] RBAC: Authorizing `identity:create_grant(project_id=98438cec810d42cab9d73e2e3170622b, user_id=8bf88c3f52964aeab9d3778bcfb6cb6e, role_id=d009411496b8452b88e77ffb8db015a7)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:29:52.718688 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f337124-bcd8-41fc-9431-59ba7b864a2c admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:29:52.732749 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1269/2534] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 01:29:52 2022] PUT /identity/v3/projects/98438cec810d42cab9d73e2e3170622b/users/8bf88c3f52964aeab9d3778bcfb6cb6e/roles/d009411496b8452b88e77ffb8db015a7 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:29:52.745405 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62535ae-723f-413c-816d-0b326e48e35f None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:29:52.745701 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62535ae-723f-413c-816d-0b326e48e35f None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:29:52.746004 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62535ae-723f-413c-816d-0b326e48e35f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:29:52.784524 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-b62535ae-723f-413c-816d-0b326e48e35f None None] MFA Rules not processed for user `8bf88c3f52964aeab9d3778bcfb6cb6e`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:29:52.788911 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-b62535ae-723f-413c-816d-0b326e48e35f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:29:52.862579 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1266/2535] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:29:52 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:29:52.874213 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f337124-bcd8-41fc-9431-59ba7b864a2c admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:29:52.880426 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-2f337124-bcd8-41fc-9431-59ba7b864a2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:29:52.913405 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:29:52.944342 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f337124-bcd8-41fc-9431-59ba7b864a2c admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:29:52.948877 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5237b553-2571-4662-8bc7-08ab26ec0603 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5c8fc69c518412ba1aad61a9f4f558c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:29:52.949485 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5237b553-2571-4662-8bc7-08ab26ec0603 service cinder] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:29:52.949768 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5237b553-2571-4662-8bc7-08ab26ec0603 service cinder] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:29:52.950043 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5237b553-2571-4662-8bc7-08ab26ec0603 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:29:52.959570 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5237b553-2571-4662-8bc7-08ab26ec0603 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:29:52.961438 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5237b553-2571-4662-8bc7-08ab26ec0603 service cinder] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:29:52.970875 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1270/2536] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:29:52 2022] GET /identity/v3/auth/tokens => generated 3347 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:29:53.105921 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62535ae-723f-413c-816d-0b326e48e35f None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:29:53.168588 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62535ae-723f-413c-816d-0b326e48e35f None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:29:53.173059 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6d5041-3fec-492b-a0f4-a4f758c366aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:29:53.173686 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6d5041-3fec-492b-a0f4-a4f758c366aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:29:53.173968 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6d5041-3fec-492b-a0f4-a4f758c366aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:29:53.174295 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6d5041-3fec-492b-a0f4-a4f758c366aa admin admin] PATH_INFO: `/v3/users/2a955d28f42345e1917f3c21d1f1ec53` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:29:53.175404 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6d5041-3fec-492b-a0f4-a4f758c366aa admin admin] RBAC: Authorizing `identity:delete_user(user_id=2a955d28f42345e1917f3c21d1f1ec53)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:29:53.177367 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6d5041-3fec-492b-a0f4-a4f758c366aa admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:29:53.350079 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-ae6d5041-3fec-492b-a0f4-a4f758c366aa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2a955d28f42345e1917f3c21d1f1ec53'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:29:53.427519 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-ae6d5041-3fec-492b-a0f4-a4f758c366aa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2a955d28f42345e1917f3c21d1f1ec53'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:29:53.428051 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-ae6d5041-3fec-492b-a0f4-a4f758c366aa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2a955d28f42345e1917f3c21d1f1ec53'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:29:53.436235 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-ae6d5041-3fec-492b-a0f4-a4f758c366aa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2a955d28f42345e1917f3c21d1f1ec53'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:29:53.436906 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-ae6d5041-3fec-492b-a0f4-a4f758c366aa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2a955d28f42345e1917f3c21d1f1ec53'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:29:53.444689 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:29:53.459168 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1267/2537] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 01:29:53 2022] DELETE /identity/v3/users/2a955d28f42345e1917f3c21d1f1ec53 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:29:53.466436 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcb6371-afc3-40df-8443-7b06fb32c220 None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:29:53.466725 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcb6371-afc3-40df-8443-7b06fb32c220 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:29:53.467003 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcb6371-afc3-40df-8443-7b06fb32c220 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:29:53.490448 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-bbcb6371-afc3-40df-8443-7b06fb32c220 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:29:53.492089 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-bbcb6371-afc3-40df-8443-7b06fb32c220 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:29:53.552234 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1271/2538] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:29:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:29:53.569005 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6d5041-3fec-492b-a0f4-a4f758c366aa admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:29:53.569005 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-ae6d5041-3fec-492b-a0f4-a4f758c366aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:29:53.624773 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6d5041-3fec-492b-a0f4-a4f758c366aa admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:29:53.628366 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745c2d83-56b6-4597-9958-54a57afa720c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0817c3320d5d41d8bba0550c8ae8dd60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:29:53.629027 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745c2d83-56b6-4597-9958-54a57afa720c service neutron] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:29:53.629310 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745c2d83-56b6-4597-9958-54a57afa720c service neutron] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:29:53.629586 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745c2d83-56b6-4597-9958-54a57afa720c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:29:53.639152 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745c2d83-56b6-4597-9958-54a57afa720c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:29:53.640605 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745c2d83-56b6-4597-9958-54a57afa720c service neutron] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:29:53.650636 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1268/2539] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:29:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:29:54.101875 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcb6371-afc3-40df-8443-7b06fb32c220 None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:29:54.103621 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-bbcb6371-afc3-40df-8443-7b06fb32c220 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:29:54.119737 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcb6371-afc3-40df-8443-7b06fb32c220 None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:29:54.125263 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd107f6-7a92-47ee-a5cd-7320b243308b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:29:54.125975 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd107f6-7a92-47ee-a5cd-7320b243308b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:29:54.127916 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd107f6-7a92-47ee-a5cd-7320b243308b admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:29:54.127916 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd107f6-7a92-47ee-a5cd-7320b243308b admin admin] PATH_INFO: `/v3/projects/aba5b6a5a4374067a75e5c4c4556072e` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:29:54.127916 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd107f6-7a92-47ee-a5cd-7320b243308b admin admin] RBAC: Authorizing `identity:delete_project(project_id=aba5b6a5a4374067a75e5c4c4556072e)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:29:54.131671 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd107f6-7a92-47ee-a5cd-7320b243308b admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:29:54.258004 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-cbd107f6-7a92-47ee-a5cd-7320b243308b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aba5b6a5a4374067a75e5c4c4556072e', 'request_id': 'req-cbd107f6-7a92-47ee-a5cd-7320b243308b'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:29:54.265894 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-cbd107f6-7a92-47ee-a5cd-7320b243308b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aba5b6a5a4374067a75e5c4c4556072e', 'request_id': 'req-cbd107f6-7a92-47ee-a5cd-7320b243308b'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:29:54.341979 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-cbd107f6-7a92-47ee-a5cd-7320b243308b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aba5b6a5a4374067a75e5c4c4556072e', 'request_id': 'req-cbd107f6-7a92-47ee-a5cd-7320b243308b'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:29:54.353667 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-cbd107f6-7a92-47ee-a5cd-7320b243308b admin admin] The token cache is being invalidate because project aba5b6a5a4374067a75e5c4c4556072e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81750) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 01:29:54.353960 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-cbd107f6-7a92-47ee-a5cd-7320b243308b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:29:54.354651 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-cbd107f6-7a92-47ee-a5cd-7320b243308b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:29:54.356101 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1272/2540] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 01:29:54 2022] DELETE /identity/v3/projects/aba5b6a5a4374067a75e5c4c4556072e => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:29:54.360207 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745c2d83-56b6-4597-9958-54a57afa720c service neutron] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:29:54.370629 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745c2d83-56b6-4597-9958-54a57afa720c service neutron] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:29:54.433410 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd80f91-3f96-4457-8077-47523731fc0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:29:54.434039 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd80f91-3f96-4457-8077-47523731fc0a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:29:54.434325 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd80f91-3f96-4457-8077-47523731fc0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:29:54.434603 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd80f91-3f96-4457-8077-47523731fc0a admin admin] PATH_INFO: `/v3/users/8bf88c3f52964aeab9d3778bcfb6cb6e` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:29:54.435749 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd80f91-3f96-4457-8077-47523731fc0a admin admin] RBAC: Authorizing `identity:delete_user(user_id=8bf88c3f52964aeab9d3778bcfb6cb6e)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:29:54.437718 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd80f91-3f96-4457-8077-47523731fc0a admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:29:54.682082 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-2fd80f91-3f96-4457-8077-47523731fc0a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8bf88c3f52964aeab9d3778bcfb6cb6e'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:29:54.792781 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-2fd80f91-3f96-4457-8077-47523731fc0a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8bf88c3f52964aeab9d3778bcfb6cb6e'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:29:54.793446 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-2fd80f91-3f96-4457-8077-47523731fc0a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8bf88c3f52964aeab9d3778bcfb6cb6e'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:29:54.800810 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-2fd80f91-3f96-4457-8077-47523731fc0a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8bf88c3f52964aeab9d3778bcfb6cb6e'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:29:54.801457 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-2fd80f91-3f96-4457-8077-47523731fc0a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8bf88c3f52964aeab9d3778bcfb6cb6e'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:29:54.820838 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1269/2541] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 01:29:54 2022] DELETE /identity/v3/users/8bf88c3f52964aeab9d3778bcfb6cb6e => generated 0 bytes in 460 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:29:55.190314 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd107f6-7a92-47ee-a5cd-7320b243308b admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:29:55.190789 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-cbd107f6-7a92-47ee-a5cd-7320b243308b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:29:55.204319 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd107f6-7a92-47ee-a5cd-7320b243308b admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:29:55.264767 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:29:55.278807 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3700d78f-14f1-409c-a634-8701dc1ff6ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:29:55.279457 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3700d78f-14f1-409c-a634-8701dc1ff6ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:29:55.279734 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3700d78f-14f1-409c-a634-8701dc1ff6ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:29:55.280011 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3700d78f-14f1-409c-a634-8701dc1ff6ea admin admin] PATH_INFO: `/v3/projects/98438cec810d42cab9d73e2e3170622b` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:29:55.281248 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3700d78f-14f1-409c-a634-8701dc1ff6ea admin admin] RBAC: Authorizing `identity:delete_project(project_id=98438cec810d42cab9d73e2e3170622b)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:29:55.283162 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3700d78f-14f1-409c-a634-8701dc1ff6ea admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:29:55.329453 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd80f91-3f96-4457-8077-47523731fc0a admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:29:55.331209 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-2fd80f91-3f96-4457-8077-47523731fc0a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:29:55.345269 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd80f91-3f96-4457-8077-47523731fc0a admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:29:55.350071 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2095539a-1d60-450c-8ba6-3b5aa7e9f855 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:29:55.350701 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2095539a-1d60-450c-8ba6-3b5aa7e9f855 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:29:55.351027 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2095539a-1d60-450c-8ba6-3b5aa7e9f855 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:29:55.351308 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2095539a-1d60-450c-8ba6-3b5aa7e9f855 admin admin] PATH_INFO: `/v3/users/d963744868964b399657c16edd212111` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:29:55.352424 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2095539a-1d60-450c-8ba6-3b5aa7e9f855 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d963744868964b399657c16edd212111)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:29:55.354424 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2095539a-1d60-450c-8ba6-3b5aa7e9f855 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:29:55.360263 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:29:55.405189 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-3700d78f-14f1-409c-a634-8701dc1ff6ea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '98438cec810d42cab9d73e2e3170622b', 'request_id': 'req-3700d78f-14f1-409c-a634-8701dc1ff6ea'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:29:55.410671 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-3700d78f-14f1-409c-a634-8701dc1ff6ea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '98438cec810d42cab9d73e2e3170622b', 'request_id': 'req-3700d78f-14f1-409c-a634-8701dc1ff6ea'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:29:55.501490 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-3700d78f-14f1-409c-a634-8701dc1ff6ea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '98438cec810d42cab9d73e2e3170622b', 'request_id': 'req-3700d78f-14f1-409c-a634-8701dc1ff6ea'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:29:55.513273 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-3700d78f-14f1-409c-a634-8701dc1ff6ea admin admin] The token cache is being invalidate because project 98438cec810d42cab9d73e2e3170622b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81750) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 01:29:55.513569 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-3700d78f-14f1-409c-a634-8701dc1ff6ea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:29:55.514192 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-3700d78f-14f1-409c-a634-8701dc1ff6ea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:29:55.515666 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1273/2542] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 01:29:55 2022] DELETE /identity/v3/projects/98438cec810d42cab9d73e2e3170622b => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:29:55.520409 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cafe6e6-0d00-474b-8ffd-67da92a60993 None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:29:55.520748 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cafe6e6-0d00-474b-8ffd-67da92a60993 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:29:55.521044 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cafe6e6-0d00-474b-8ffd-67da92a60993 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:29:55.540366 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-7cafe6e6-0d00-474b-8ffd-67da92a60993 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:29:55.541613 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-7cafe6e6-0d00-474b-8ffd-67da92a60993 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:29:55.601547 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1274/2543] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:29:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 01:29:55.609478 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1366db1-9de7-4183-a2ee-6e2b6137531b None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:29:55.609773 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1366db1-9de7-4183-a2ee-6e2b6137531b None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:29:55.610057 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1366db1-9de7-4183-a2ee-6e2b6137531b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:29:55.612294 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-2095539a-1d60-450c-8ba6-3b5aa7e9f855 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd963744868964b399657c16edd212111'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:29:55.629320 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-b1366db1-9de7-4183-a2ee-6e2b6137531b None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:29:55.630536 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-b1366db1-9de7-4183-a2ee-6e2b6137531b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:29:55.639062 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1275/2544] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:29:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 01:29:55.643135 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1366db1-9de7-4183-a2ee-6e2b6137531b None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:29:55.652013 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1366db1-9de7-4183-a2ee-6e2b6137531b None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:29:55.658560 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043e9489-fd1c-4592-a9c1-b1de3f511979 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:29:55.658560 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043e9489-fd1c-4592-a9c1-b1de3f511979 admin admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:29:55.658560 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043e9489-fd1c-4592-a9c1-b1de3f511979 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:29:55.658560 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043e9489-fd1c-4592-a9c1-b1de3f511979 admin admin] PATH_INFO: `/v3/domains` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:29:55.658953 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043e9489-fd1c-4592-a9c1-b1de3f511979 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:29:55.658953 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043e9489-fd1c-4592-a9c1-b1de3f511979 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:29:55.660507 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043e9489-fd1c-4592-a9c1-b1de3f511979 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:29:55.672516 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1276/2545] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Aug 31 01:29:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 01:29:55.676538 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043e9489-fd1c-4592-a9c1-b1de3f511979 admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:29:55.686862 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043e9489-fd1c-4592-a9c1-b1de3f511979 admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:29:55.691590 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b17a93-df9f-40aa-8e1f-58bf9a43bbc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:29:55.692358 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b17a93-df9f-40aa-8e1f-58bf9a43bbc2 admin admin] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:29:55.692649 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b17a93-df9f-40aa-8e1f-58bf9a43bbc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:29:55.692962 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b17a93-df9f-40aa-8e1f-58bf9a43bbc2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:29:55.693638 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b17a93-df9f-40aa-8e1f-58bf9a43bbc2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:29:55.695577 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b17a93-df9f-40aa-8e1f-58bf9a43bbc2 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:29:55.710513 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-2095539a-1d60-450c-8ba6-3b5aa7e9f855 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd963744868964b399657c16edd212111'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:29:55.712332 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-2095539a-1d60-450c-8ba6-3b5aa7e9f855 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd963744868964b399657c16edd212111'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:29:55.724956 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-2095539a-1d60-450c-8ba6-3b5aa7e9f855 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd963744868964b399657c16edd212111'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:29:55.725814 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-2095539a-1d60-450c-8ba6-3b5aa7e9f855 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd963744868964b399657c16edd212111'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:29:55.750672 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1270/2546] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 01:29:55 2022] DELETE /identity/v3/users/d963744868964b399657c16edd212111 => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 01:29:55.758762 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5dc01d-6b10-407a-93aa-53978d237bd8 None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:29:55.759075 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5dc01d-6b10-407a-93aa-53978d237bd8 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:29:55.759387 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5dc01d-6b10-407a-93aa-53978d237bd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:29:55.778827 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-7a5dc01d-6b10-407a-93aa-53978d237bd8 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:29:55.780062 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-7a5dc01d-6b10-407a-93aa-53978d237bd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:29:55.797120 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1277/2547] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 01:29:55 2022] POST /identity/v3/projects => generated 381 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 01:29:55.800973 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b17a93-df9f-40aa-8e1f-58bf9a43bbc2 admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:29:55.802702 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-a6b17a93-df9f-40aa-8e1f-58bf9a43bbc2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:29:55.842919 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:29:55.843877 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1271/2548] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:29:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 01:29:55.854375 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5dc01d-6b10-407a-93aa-53978d237bd8 None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:29:55.855936 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-7a5dc01d-6b10-407a-93aa-53978d237bd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:29:55.867601 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b17a93-df9f-40aa-8e1f-58bf9a43bbc2 admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:29:55.872471 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee34a25-0568-4112-9e40-c7947fb21515 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:29:55.873123 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee34a25-0568-4112-9e40-c7947fb21515 admin admin] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:29:55.873409 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee34a25-0568-4112-9e40-c7947fb21515 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:29:55.873687 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee34a25-0568-4112-9e40-c7947fb21515 admin admin] PATH_INFO: `/v3/users` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:29:55.874343 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee34a25-0568-4112-9e40-c7947fb21515 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:29:55.876276 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee34a25-0568-4112-9e40-c7947fb21515 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:29:55.915198 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5dc01d-6b10-407a-93aa-53978d237bd8 None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:29:55.953541 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:29:55.962416 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740d2712-c8b7-42ba-ac79-30588ec1a686 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0817c3320d5d41d8bba0550c8ae8dd60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:29:55.963605 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740d2712-c8b7-42ba-ac79-30588ec1a686 service neutron] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:29:55.963891 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740d2712-c8b7-42ba-ac79-30588ec1a686 service neutron] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:29:55.964169 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740d2712-c8b7-42ba-ac79-30588ec1a686 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:29:55.965100 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1278/2549] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 01:29:55 2022] POST /identity/v3/users => generated 291 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 01:29:55.969444 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee34a25-0568-4112-9e40-c7947fb21515 admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:29:55.977017 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740d2712-c8b7-42ba-ac79-30588ec1a686 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:29:55.981918 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740d2712-c8b7-42ba-ac79-30588ec1a686 service neutron] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:29:55.982031 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee34a25-0568-4112-9e40-c7947fb21515 admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:29:55.984998 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7212f89-114e-4f3b-a131-972ba52b6ed2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:29:55.985531 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7212f89-114e-4f3b-a131-972ba52b6ed2 admin admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:29:55.985856 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7212f89-114e-4f3b-a131-972ba52b6ed2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:29:55.986128 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7212f89-114e-4f3b-a131-972ba52b6ed2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:29:55.986756 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7212f89-114e-4f3b-a131-972ba52b6ed2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:29:55.988328 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7212f89-114e-4f3b-a131-972ba52b6ed2 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:29:55.989044 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1272/2550] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:29:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 01:29:55.998580 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1279/2551] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:29:55 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 01:29:56.002467 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740d2712-c8b7-42ba-ac79-30588ec1a686 service neutron] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:29:56.012139 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740d2712-c8b7-42ba-ac79-30588ec1a686 service neutron] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:29:56.016877 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133ad77f-89fe-447b-9d94-5a79128bddb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:29:56.017581 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133ad77f-89fe-447b-9d94-5a79128bddb5 admin admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:29:56.018251 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133ad77f-89fe-447b-9d94-5a79128bddb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:29:56.018251 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133ad77f-89fe-447b-9d94-5a79128bddb5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:29:56.018879 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133ad77f-89fe-447b-9d94-5a79128bddb5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:29:56.020575 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133ad77f-89fe-447b-9d94-5a79128bddb5 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:29:56.033131 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1273/2552] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:29:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:29:56.036676 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7212f89-114e-4f3b-a131-972ba52b6ed2 admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:29:56.047570 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7212f89-114e-4f3b-a131-972ba52b6ed2 admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:29:56.052905 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d16322-aca3-4836-b851-17a3040e259e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:29:56.053664 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d16322-aca3-4836-b851-17a3040e259e admin admin] REQUEST_METHOD: `PUT` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:29:56.054021 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d16322-aca3-4836-b851-17a3040e259e admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:29:56.054372 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d16322-aca3-4836-b851-17a3040e259e admin admin] PATH_INFO: `/v3/projects/10d98f3c49c54c9c8080b6d2ba96d6a2/users/f42e604eb8fd43618e7f388cfeb4efca/roles/d0b9768f4d014766b9326387f2e11f34` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:29:56.069831 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d16322-aca3-4836-b851-17a3040e259e admin admin] RBAC: Authorizing `identity:create_grant(project_id=10d98f3c49c54c9c8080b6d2ba96d6a2, user_id=f42e604eb8fd43618e7f388cfeb4efca, role_id=d0b9768f4d014766b9326387f2e11f34)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:29:56.073107 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d16322-aca3-4836-b851-17a3040e259e admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:29:56.141415 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1280/2553] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 01:29:56 2022] PUT /identity/v3/projects/10d98f3c49c54c9c8080b6d2ba96d6a2/users/f42e604eb8fd43618e7f388cfeb4efca/roles/d0b9768f4d014766b9326387f2e11f34 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 01:29:56.151627 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1352606-8bb7-4617-8fe0-7b607f5efe22 None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:29:56.151919 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1352606-8bb7-4617-8fe0-7b607f5efe22 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:29:56.152198 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1352606-8bb7-4617-8fe0-7b607f5efe22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:29:56.182324 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-d1352606-8bb7-4617-8fe0-7b607f5efe22 None None] MFA Rules not processed for user `f42e604eb8fd43618e7f388cfeb4efca`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:29:56.183799 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-d1352606-8bb7-4617-8fe0-7b607f5efe22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:29:56.251531 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1274/2554] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:29:56 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:29:56.255939 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d16322-aca3-4836-b851-17a3040e259e admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:29:56.321723 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d16322-aca3-4836-b851-17a3040e259e admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:29:56.326786 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bc9e9d-b634-4eb2-a180-fec582e9b8fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:29:56.327670 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bc9e9d-b634-4eb2-a180-fec582e9b8fe admin admin] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:29:56.328011 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bc9e9d-b634-4eb2-a180-fec582e9b8fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:29:56.328295 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bc9e9d-b634-4eb2-a180-fec582e9b8fe admin admin] PATH_INFO: `/v3/projects` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:29:56.329032 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bc9e9d-b634-4eb2-a180-fec582e9b8fe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:29:56.331676 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bc9e9d-b634-4eb2-a180-fec582e9b8fe admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:29:56.401363 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1352606-8bb7-4617-8fe0-7b607f5efe22 None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:29:56.402838 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-d1352606-8bb7-4617-8fe0-7b607f5efe22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:29:56.417197 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1352606-8bb7-4617-8fe0-7b607f5efe22 None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:29:56.422521 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1592026e-6efa-49a6-86dd-3ba4ebc48418 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:29:56.423168 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1592026e-6efa-49a6-86dd-3ba4ebc48418 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:29:56.423455 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1592026e-6efa-49a6-86dd-3ba4ebc48418 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:29:56.423794 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1592026e-6efa-49a6-86dd-3ba4ebc48418 admin admin] PATH_INFO: `/v3/projects/bb74d110d646466794ccb97d204e1480` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:29:56.424976 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1592026e-6efa-49a6-86dd-3ba4ebc48418 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bb74d110d646466794ccb97d204e1480)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:29:56.427020 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1592026e-6efa-49a6-86dd-3ba4ebc48418 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:29:56.432232 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1281/2555] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 01:29:56 2022] POST /identity/v3/projects => generated 379 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:29:56.433311 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bc9e9d-b634-4eb2-a180-fec582e9b8fe admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:29:56.509828 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bc9e9d-b634-4eb2-a180-fec582e9b8fe admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:29:56.515131 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70432a71-0733-42ab-8a84-e3f5c1156c13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:29:56.515526 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70432a71-0733-42ab-8a84-e3f5c1156c13 admin admin] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:29:56.515812 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70432a71-0733-42ab-8a84-e3f5c1156c13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:29:56.516117 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70432a71-0733-42ab-8a84-e3f5c1156c13 admin admin] PATH_INFO: `/v3/users` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:29:56.516847 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70432a71-0733-42ab-8a84-e3f5c1156c13 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:29:56.518954 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70432a71-0733-42ab-8a84-e3f5c1156c13 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:29:56.539753 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-1592026e-6efa-49a6-86dd-3ba4ebc48418 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bb74d110d646466794ccb97d204e1480', 'request_id': 'req-1592026e-6efa-49a6-86dd-3ba4ebc48418'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:29:56.546297 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-1592026e-6efa-49a6-86dd-3ba4ebc48418 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bb74d110d646466794ccb97d204e1480', 'request_id': 'req-1592026e-6efa-49a6-86dd-3ba4ebc48418'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:29:56.552421 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-1592026e-6efa-49a6-86dd-3ba4ebc48418 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bb74d110d646466794ccb97d204e1480', 'request_id': 'req-1592026e-6efa-49a6-86dd-3ba4ebc48418'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:29:56.638701 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1282/2556] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 01:29:56 2022] POST /identity/v3/users => generated 296 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 01:29:56.642781 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70432a71-0733-42ab-8a84-e3f5c1156c13 admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:29:56.648752 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:29:56.653310 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70432a71-0733-42ab-8a84-e3f5c1156c13 admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:29:56.655602 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-1592026e-6efa-49a6-86dd-3ba4ebc48418 admin admin] The token cache is being invalidate because project bb74d110d646466794ccb97d204e1480 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81749) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 01:29:56.655898 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-1592026e-6efa-49a6-86dd-3ba4ebc48418 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:29:56.656528 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-1592026e-6efa-49a6-86dd-3ba4ebc48418 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:29:56.658072 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1275/2557] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 01:29:56 2022] DELETE /identity/v3/projects/bb74d110d646466794ccb97d204e1480 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:29:56.662016 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1592026e-6efa-49a6-86dd-3ba4ebc48418 admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:29:56.663402 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-1592026e-6efa-49a6-86dd-3ba4ebc48418 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:29:56.714834 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:29:56.715874 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eef040c-2136-4cda-bec4-948a9c8e0c59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:29:56.716553 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eef040c-2136-4cda-bec4-948a9c8e0c59 admin admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:29:56.716871 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eef040c-2136-4cda-bec4-948a9c8e0c59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:29:56.717177 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eef040c-2136-4cda-bec4-948a9c8e0c59 admin admin] PATH_INFO: `/v3/roles` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:29:56.717816 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eef040c-2136-4cda-bec4-948a9c8e0c59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:29:56.719813 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eef040c-2136-4cda-bec4-948a9c8e0c59 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:29:56.730464 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1283/2558] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:29:56 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 01:29:56.731230 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1592026e-6efa-49a6-86dd-3ba4ebc48418 admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:29:56.734739 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eef040c-2136-4cda-bec4-948a9c8e0c59 admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:29:56.736502 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-7eef040c-2136-4cda-bec4-948a9c8e0c59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:29:56.736626 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c55efa9-d1fc-4873-9fc7-96be0a50e85f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:29:56.737324 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c55efa9-d1fc-4873-9fc7-96be0a50e85f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:29:56.737671 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c55efa9-d1fc-4873-9fc7-96be0a50e85f admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:29:56.737935 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c55efa9-d1fc-4873-9fc7-96be0a50e85f admin admin] PATH_INFO: `/v3/users/3351f082868b4ff2b9688cdd49dcbdc0` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:29:56.739072 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c55efa9-d1fc-4873-9fc7-96be0a50e85f admin admin] RBAC: Authorizing `identity:delete_user(user_id=3351f082868b4ff2b9688cdd49dcbdc0)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:29:56.742924 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c55efa9-d1fc-4873-9fc7-96be0a50e85f admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:29:56.749405 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eef040c-2136-4cda-bec4-948a9c8e0c59 admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:29:56.754742 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544e2af0-a26e-4d1a-992a-ab6872e7b3d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:29:56.755358 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544e2af0-a26e-4d1a-992a-ab6872e7b3d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:29:56.755640 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544e2af0-a26e-4d1a-992a-ab6872e7b3d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:29:56.755916 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544e2af0-a26e-4d1a-992a-ab6872e7b3d0 admin admin] PATH_INFO: `/v3/projects/5f0b244db4db41baba47aefc52cb4c7b/users/677cf03698bd424282e7a043d33ed921/roles/d009411496b8452b88e77ffb8db015a7` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:29:56.775943 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544e2af0-a26e-4d1a-992a-ab6872e7b3d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f0b244db4db41baba47aefc52cb4c7b, user_id=677cf03698bd424282e7a043d33ed921, role_id=d009411496b8452b88e77ffb8db015a7)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:29:56.778651 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544e2af0-a26e-4d1a-992a-ab6872e7b3d0 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:29:56.851479 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1284/2559] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 01:29:56 2022] PUT /identity/v3/projects/5f0b244db4db41baba47aefc52cb4c7b/users/677cf03698bd424282e7a043d33ed921/roles/d009411496b8452b88e77ffb8db015a7 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:29:56.855238 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544e2af0-a26e-4d1a-992a-ab6872e7b3d0 admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:29:56.918607 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-6c55efa9-d1fc-4873-9fc7-96be0a50e85f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3351f082868b4ff2b9688cdd49dcbdc0'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:29:56.926567 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544e2af0-a26e-4d1a-992a-ab6872e7b3d0 admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:29:56.931893 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a8eb0b-92c2-494d-a3a9-b444662de175 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:29:56.932508 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a8eb0b-92c2-494d-a3a9-b444662de175 admin admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:29:56.932848 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a8eb0b-92c2-494d-a3a9-b444662de175 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:29:56.933126 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a8eb0b-92c2-494d-a3a9-b444662de175 admin admin] PATH_INFO: `/v3/roles` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:29:56.933769 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a8eb0b-92c2-494d-a3a9-b444662de175 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:29:56.935922 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a8eb0b-92c2-494d-a3a9-b444662de175 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:29:56.946425 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1285/2560] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:29:56 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 01:29:56.951183 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a8eb0b-92c2-494d-a3a9-b444662de175 admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:29:56.960947 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a8eb0b-92c2-494d-a3a9-b444662de175 admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:29:56.966269 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6e3330-338d-4c7b-b63e-cc668ff689e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:29:56.966902 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6e3330-338d-4c7b-b63e-cc668ff689e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:29:56.967182 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6e3330-338d-4c7b-b63e-cc668ff689e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:29:56.967457 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6e3330-338d-4c7b-b63e-cc668ff689e9 admin admin] PATH_INFO: `/v3/projects/5f0b244db4db41baba47aefc52cb4c7b/users/677cf03698bd424282e7a043d33ed921/roles/d009411496b8452b88e77ffb8db015a7` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:29:56.969443 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6e3330-338d-4c7b-b63e-cc668ff689e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f0b244db4db41baba47aefc52cb4c7b, user_id=677cf03698bd424282e7a043d33ed921, role_id=d009411496b8452b88e77ffb8db015a7)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:29:56.972107 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6e3330-338d-4c7b-b63e-cc668ff689e9 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:29:56.981468 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-6c55efa9-d1fc-4873-9fc7-96be0a50e85f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3351f082868b4ff2b9688cdd49dcbdc0'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:29:56.982108 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-6c55efa9-d1fc-4873-9fc7-96be0a50e85f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3351f082868b4ff2b9688cdd49dcbdc0'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:29:56.986795 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1286/2561] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 01:29:56 2022] PUT /identity/v3/projects/5f0b244db4db41baba47aefc52cb4c7b/users/677cf03698bd424282e7a043d33ed921/roles/d009411496b8452b88e77ffb8db015a7 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 01:29:56.990475 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-6c55efa9-d1fc-4873-9fc7-96be0a50e85f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3351f082868b4ff2b9688cdd49dcbdc0'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:29:56.991273 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-6c55efa9-d1fc-4873-9fc7-96be0a50e85f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3351f082868b4ff2b9688cdd49dcbdc0'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:29:56.999015 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0550cab6-ee4e-4530-bd0b-16c4dec8f3bd None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:29:56.999497 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0550cab6-ee4e-4530-bd0b-16c4dec8f3bd None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:29:56.999955 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0550cab6-ee4e-4530-bd0b-16c4dec8f3bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:29:57.015937 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1276/2562] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 01:29:56 2022] DELETE /identity/v3/users/3351f082868b4ff2b9688cdd49dcbdc0 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 01:29:57.030907 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-0550cab6-ee4e-4530-bd0b-16c4dec8f3bd None None] MFA Rules not processed for user `677cf03698bd424282e7a043d33ed921`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:29:57.032147 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-0550cab6-ee4e-4530-bd0b-16c4dec8f3bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:29:57.080600 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:29:57.103954 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1287/2563] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:29:56 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:29:57.120612 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c55efa9-d1fc-4873-9fc7-96be0a50e85f admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:29:57.122092 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-6c55efa9-d1fc-4873-9fc7-96be0a50e85f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:29:57.179476 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c55efa9-d1fc-4873-9fc7-96be0a50e85f admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:29:57.189682 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b4ca0a-1662-4af8-916d-d5fb1b889531 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5c8fc69c518412ba1aad61a9f4f558c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:29:57.190322 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b4ca0a-1662-4af8-916d-d5fb1b889531 service cinder] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:29:57.190609 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b4ca0a-1662-4af8-916d-d5fb1b889531 service cinder] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:29:57.190889 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b4ca0a-1662-4af8-916d-d5fb1b889531 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:29:57.201729 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b4ca0a-1662-4af8-916d-d5fb1b889531 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:29:57.203294 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b4ca0a-1662-4af8-916d-d5fb1b889531 service cinder] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:29:57.215160 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1277/2564] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:29:57 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 01:29:57.430953 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0550cab6-ee4e-4530-bd0b-16c4dec8f3bd None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:29:57.433066 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-0550cab6-ee4e-4530-bd0b-16c4dec8f3bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:29:57.527970 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0550cab6-ee4e-4530-bd0b-16c4dec8f3bd None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:29:57.532481 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955d48a3-4fd1-4e5e-833b-827ce7e84c87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:29:57.533142 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955d48a3-4fd1-4e5e-833b-827ce7e84c87 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:29:57.533428 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955d48a3-4fd1-4e5e-833b-827ce7e84c87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:29:57.533722 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955d48a3-4fd1-4e5e-833b-827ce7e84c87 admin admin] PATH_INFO: `/v3/projects/d2cbab83125a4121b2d4f1ca6a987373` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:29:57.534875 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955d48a3-4fd1-4e5e-833b-827ce7e84c87 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d2cbab83125a4121b2d4f1ca6a987373)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:29:57.536805 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955d48a3-4fd1-4e5e-833b-827ce7e84c87 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:29:57.639678 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-955d48a3-4fd1-4e5e-833b-827ce7e84c87 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd2cbab83125a4121b2d4f1ca6a987373', 'request_id': 'req-955d48a3-4fd1-4e5e-833b-827ce7e84c87'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:29:57.645937 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-955d48a3-4fd1-4e5e-833b-827ce7e84c87 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd2cbab83125a4121b2d4f1ca6a987373', 'request_id': 'req-955d48a3-4fd1-4e5e-833b-827ce7e84c87'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:29:57.730285 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-955d48a3-4fd1-4e5e-833b-827ce7e84c87 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd2cbab83125a4121b2d4f1ca6a987373', 'request_id': 'req-955d48a3-4fd1-4e5e-833b-827ce7e84c87'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:29:57.742067 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-955d48a3-4fd1-4e5e-833b-827ce7e84c87 admin admin] The token cache is being invalidate because project d2cbab83125a4121b2d4f1ca6a987373 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81750) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 01:29:57.742359 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-955d48a3-4fd1-4e5e-833b-827ce7e84c87 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:29:57.742996 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-955d48a3-4fd1-4e5e-833b-827ce7e84c87 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:29:57.745093 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1288/2565] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 01:29:57 2022] DELETE /identity/v3/projects/d2cbab83125a4121b2d4f1ca6a987373 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:29:57.749954 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ec4354-e95a-4d0e-9780-eaea8a210e76 None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:29:57.750436 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ec4354-e95a-4d0e-9780-eaea8a210e76 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:29:57.750532 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ec4354-e95a-4d0e-9780-eaea8a210e76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:29:57.769607 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-63ec4354-e95a-4d0e-9780-eaea8a210e76 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:29:57.770797 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-63ec4354-e95a-4d0e-9780-eaea8a210e76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:29:57.831508 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1278/2566] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:29:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:29:57.839393 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424275c3-732f-4289-a58c-daf746a82147 None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:29:57.839677 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424275c3-732f-4289-a58c-daf746a82147 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:29:57.839988 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424275c3-732f-4289-a58c-daf746a82147 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:29:57.859821 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-424275c3-732f-4289-a58c-daf746a82147 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:29:57.861133 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-424275c3-732f-4289-a58c-daf746a82147 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:29:57.869855 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1289/2567] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:29:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:29:57.883635 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ec4354-e95a-4d0e-9780-eaea8a210e76 None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:29:57.893382 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ec4354-e95a-4d0e-9780-eaea8a210e76 None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:29:57.903189 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f13e6b-acaa-4276-b5b7-92f39a4bdef4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:29:57.903845 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f13e6b-acaa-4276-b5b7-92f39a4bdef4 admin admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:29:57.904129 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f13e6b-acaa-4276-b5b7-92f39a4bdef4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:29:57.907920 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f13e6b-acaa-4276-b5b7-92f39a4bdef4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:29:57.907920 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f13e6b-acaa-4276-b5b7-92f39a4bdef4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:29:57.907920 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f13e6b-acaa-4276-b5b7-92f39a4bdef4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:29:57.907920 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f13e6b-acaa-4276-b5b7-92f39a4bdef4 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:29:57.909875 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:29:57.920203 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1279/2568] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Aug 31 01:29:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:29:57.925142 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424275c3-732f-4289-a58c-daf746a82147 None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:29:57.934704 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424275c3-732f-4289-a58c-daf746a82147 None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:29:57.939777 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547c116f-e6dd-4273-8bc0-b20a60f3100e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:29:57.940385 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547c116f-e6dd-4273-8bc0-b20a60f3100e admin admin] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:29:57.942382 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547c116f-e6dd-4273-8bc0-b20a60f3100e admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:29:57.942382 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547c116f-e6dd-4273-8bc0-b20a60f3100e admin admin] PATH_INFO: `/v3/projects` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:29:57.942382 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547c116f-e6dd-4273-8bc0-b20a60f3100e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:29:57.943577 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547c116f-e6dd-4273-8bc0-b20a60f3100e admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:29:58.034139 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1290/2569] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 01:29:57 2022] POST /identity/v3/projects => generated 389 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:29:58.038328 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f13e6b-acaa-4276-b5b7-92f39a4bdef4 admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:29:58.048747 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f13e6b-acaa-4276-b5b7-92f39a4bdef4 admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:29:58.102647 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3e27d2-44bf-4b56-8454-9b204731c518 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:29:58.103800 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3e27d2-44bf-4b56-8454-9b204731c518 admin admin] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:29:58.104084 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3e27d2-44bf-4b56-8454-9b204731c518 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:29:58.104357 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3e27d2-44bf-4b56-8454-9b204731c518 admin admin] PATH_INFO: `/v3/users` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:29:58.105022 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3e27d2-44bf-4b56-8454-9b204731c518 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:29:58.106932 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3e27d2-44bf-4b56-8454-9b204731c518 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:29:58.163819 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1280/2570] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 01:29:58 2022] POST /identity/v3/users => generated 295 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:29:58.168602 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547c116f-e6dd-4273-8bc0-b20a60f3100e admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:29:58.178990 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547c116f-e6dd-4273-8bc0-b20a60f3100e admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:29:58.183909 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c703d6f3-c58f-4259-9d4d-cbe886c1fec8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:29:58.184487 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c703d6f3-c58f-4259-9d4d-cbe886c1fec8 admin admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:29:58.187443 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c703d6f3-c58f-4259-9d4d-cbe886c1fec8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:29:58.187443 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c703d6f3-c58f-4259-9d4d-cbe886c1fec8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:29:58.187443 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c703d6f3-c58f-4259-9d4d-cbe886c1fec8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:29:58.187443 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c703d6f3-c58f-4259-9d4d-cbe886c1fec8 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:29:58.197117 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1291/2571] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:29:58 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:29:58.201711 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3e27d2-44bf-4b56-8454-9b204731c518 admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:29:58.213048 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3e27d2-44bf-4b56-8454-9b204731c518 admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:29:58.217674 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d6327b-e612-46a6-b710-e896b0356599 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:29:58.218265 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d6327b-e612-46a6-b710-e896b0356599 admin admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:29:58.218552 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d6327b-e612-46a6-b710-e896b0356599 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:29:58.218830 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d6327b-e612-46a6-b710-e896b0356599 admin admin] PATH_INFO: `/v3/roles` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:29:58.219472 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d6327b-e612-46a6-b710-e896b0356599 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:29:58.221268 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d6327b-e612-46a6-b710-e896b0356599 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:29:58.235847 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1281/2572] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:29:58 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:29:58.237116 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c703d6f3-c58f-4259-9d4d-cbe886c1fec8 admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:29:58.246982 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c703d6f3-c58f-4259-9d4d-cbe886c1fec8 admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:29:58.256322 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cdfffa-bc9e-4213-a404-89f3b4903adc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:29:58.257019 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cdfffa-bc9e-4213-a404-89f3b4903adc admin admin] REQUEST_METHOD: `PUT` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:29:58.257304 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cdfffa-bc9e-4213-a404-89f3b4903adc admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:29:58.257581 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cdfffa-bc9e-4213-a404-89f3b4903adc admin admin] PATH_INFO: `/v3/projects/f52e1b886f364391942978ef33782d97/users/d3dbce6124d24f739b61be17a5aeeb1f/roles/d0b9768f4d014766b9326387f2e11f34` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:29:58.272555 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cdfffa-bc9e-4213-a404-89f3b4903adc admin admin] RBAC: Authorizing `identity:create_grant(project_id=f52e1b886f364391942978ef33782d97, user_id=d3dbce6124d24f739b61be17a5aeeb1f, role_id=d0b9768f4d014766b9326387f2e11f34)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:29:58.275766 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cdfffa-bc9e-4213-a404-89f3b4903adc admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:29:58.291831 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d6327b-e612-46a6-b710-e896b0356599 admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:29:58.293572 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-88d6327b-e612-46a6-b710-e896b0356599 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:29:58.342407 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1292/2573] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 01:29:58 2022] PUT /identity/v3/projects/f52e1b886f364391942978ef33782d97/users/d3dbce6124d24f739b61be17a5aeeb1f/roles/d0b9768f4d014766b9326387f2e11f34 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:29:58.354488 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27bcb73-a911-4f72-b031-967c6689849f None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:29:58.354697 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27bcb73-a911-4f72-b031-967c6689849f None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:29:58.355055 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27bcb73-a911-4f72-b031-967c6689849f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:29:58.357460 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d6327b-e612-46a6-b710-e896b0356599 admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:29:58.376311 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:29:58.390002 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-d27bcb73-a911-4f72-b031-967c6689849f None None] MFA Rules not processed for user `d3dbce6124d24f739b61be17a5aeeb1f`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:29:58.391236 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-d27bcb73-a911-4f72-b031-967c6689849f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:29:58.417391 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15e2006-5bb3-40b2-a8b6-a56bc139cb89 tempest-VolumeQuotaClassesTest-958998017 tempest-VolumeQuotaClassesTest-958998017-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '677cf03698bd424282e7a043d33ed921', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f0b244db4db41baba47aefc52cb4c7b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:29:58.423394 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15e2006-5bb3-40b2-a8b6-a56bc139cb89 tempest-VolumeQuotaClassesTest-958998017 tempest-VolumeQuotaClassesTest-958998017-project-admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:29:58.423394 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15e2006-5bb3-40b2-a8b6-a56bc139cb89 tempest-VolumeQuotaClassesTest-958998017 tempest-VolumeQuotaClassesTest-958998017-project-admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:29:58.423394 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15e2006-5bb3-40b2-a8b6-a56bc139cb89 tempest-VolumeQuotaClassesTest-958998017 tempest-VolumeQuotaClassesTest-958998017-project-admin] PATH_INFO: `/v3/domains` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:29:58.423394 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15e2006-5bb3-40b2-a8b6-a56bc139cb89 tempest-VolumeQuotaClassesTest-958998017 tempest-VolumeQuotaClassesTest-958998017-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:29:58.423394 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15e2006-5bb3-40b2-a8b6-a56bc139cb89 tempest-VolumeQuotaClassesTest-958998017 tempest-VolumeQuotaClassesTest-958998017-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:29:58.423844 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15e2006-5bb3-40b2-a8b6-a56bc139cb89 tempest-VolumeQuotaClassesTest-958998017 tempest-VolumeQuotaClassesTest-958998017-project-admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:29:58.437285 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1282/2574] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Aug 31 01:29:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 01:29:58.438375 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15e2006-5bb3-40b2-a8b6-a56bc139cb89 tempest-VolumeQuotaClassesTest-958998017 tempest-VolumeQuotaClassesTest-958998017-project-admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:29:58.448807 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15e2006-5bb3-40b2-a8b6-a56bc139cb89 tempest-VolumeQuotaClassesTest-958998017 tempest-VolumeQuotaClassesTest-958998017-project-admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:29:58.453467 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99840de6-715d-4013-93a3-34a5bb821c52 tempest-VolumeQuotaClassesTest-958998017 tempest-VolumeQuotaClassesTest-958998017-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '677cf03698bd424282e7a043d33ed921', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f0b244db4db41baba47aefc52cb4c7b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:29:58.454068 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99840de6-715d-4013-93a3-34a5bb821c52 tempest-VolumeQuotaClassesTest-958998017 tempest-VolumeQuotaClassesTest-958998017-project-admin] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:29:58.454373 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99840de6-715d-4013-93a3-34a5bb821c52 tempest-VolumeQuotaClassesTest-958998017 tempest-VolumeQuotaClassesTest-958998017-project-admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:29:58.454655 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99840de6-715d-4013-93a3-34a5bb821c52 tempest-VolumeQuotaClassesTest-958998017 tempest-VolumeQuotaClassesTest-958998017-project-admin] PATH_INFO: `/v3/projects` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:29:58.455537 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99840de6-715d-4013-93a3-34a5bb821c52 tempest-VolumeQuotaClassesTest-958998017 tempest-VolumeQuotaClassesTest-958998017-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:29:58.457568 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99840de6-715d-4013-93a3-34a5bb821c52 tempest-VolumeQuotaClassesTest-958998017 tempest-VolumeQuotaClassesTest-958998017-project-admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:29:58.458069 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1293/2575] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:29:58 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:29:58.462460 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27bcb73-a911-4f72-b031-967c6689849f None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:29:58.473990 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:29:58.532558 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27bcb73-a911-4f72-b031-967c6689849f None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:29:58.537515 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8108b1f0-4b97-48a2-a491-af775aa99e4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:29:58.538186 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8108b1f0-4b97-48a2-a491-af775aa99e4f admin admin] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:29:58.538450 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8108b1f0-4b97-48a2-a491-af775aa99e4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:29:58.538726 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8108b1f0-4b97-48a2-a491-af775aa99e4f admin admin] PATH_INFO: `/v3/projects` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:29:58.539386 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8108b1f0-4b97-48a2-a491-af775aa99e4f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:29:58.541710 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8108b1f0-4b97-48a2-a491-af775aa99e4f admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:29:58.549224 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1283/2576] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 01:29:58 2022] POST /identity/v3/projects => generated 355 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:29:58.553287 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99840de6-715d-4013-93a3-34a5bb821c52 tempest-VolumeQuotaClassesTest-958998017 tempest-VolumeQuotaClassesTest-958998017-project-admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:29:58.626166 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99840de6-715d-4013-93a3-34a5bb821c52 tempest-VolumeQuotaClassesTest-958998017 tempest-VolumeQuotaClassesTest-958998017-project-admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:29:58.626166 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a395cf5-0698-461a-a863-32d17f1269ff tempest-VolumeQuotaClassesTest-958998017 tempest-VolumeQuotaClassesTest-958998017-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '677cf03698bd424282e7a043d33ed921', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f0b244db4db41baba47aefc52cb4c7b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:29:58.626166 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a395cf5-0698-461a-a863-32d17f1269ff tempest-VolumeQuotaClassesTest-958998017 tempest-VolumeQuotaClassesTest-958998017-project-admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:29:58.626809 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a395cf5-0698-461a-a863-32d17f1269ff tempest-VolumeQuotaClassesTest-958998017 tempest-VolumeQuotaClassesTest-958998017-project-admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:29:58.626809 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a395cf5-0698-461a-a863-32d17f1269ff tempest-VolumeQuotaClassesTest-958998017 tempest-VolumeQuotaClassesTest-958998017-project-admin] PATH_INFO: `/v3/domains` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:29:58.626809 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a395cf5-0698-461a-a863-32d17f1269ff tempest-VolumeQuotaClassesTest-958998017 tempest-VolumeQuotaClassesTest-958998017-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:29:58.626809 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a395cf5-0698-461a-a863-32d17f1269ff tempest-VolumeQuotaClassesTest-958998017 tempest-VolumeQuotaClassesTest-958998017-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:29:58.626809 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a395cf5-0698-461a-a863-32d17f1269ff tempest-VolumeQuotaClassesTest-958998017 tempest-VolumeQuotaClassesTest-958998017-project-admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:29:58.638098 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1284/2577] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Aug 31 01:29:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:29:58.653167 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1294/2578] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 01:29:58 2022] POST /identity/v3/projects => generated 389 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 01:29:58.657738 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a395cf5-0698-461a-a863-32d17f1269ff tempest-VolumeQuotaClassesTest-958998017 tempest-VolumeQuotaClassesTest-958998017-project-admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:29:58.686828 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8108b1f0-4b97-48a2-a491-af775aa99e4f admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:29:58.724558 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a395cf5-0698-461a-a863-32d17f1269ff tempest-VolumeQuotaClassesTest-958998017 tempest-VolumeQuotaClassesTest-958998017-project-admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:29:58.749955 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8108b1f0-4b97-48a2-a491-af775aa99e4f admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:29:58.754799 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f96f23-5663-485e-81bb-690750e06da8 tempest-VolumeQuotaClassesTest-958998017 tempest-VolumeQuotaClassesTest-958998017-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '677cf03698bd424282e7a043d33ed921', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f0b244db4db41baba47aefc52cb4c7b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:29:58.755436 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f96f23-5663-485e-81bb-690750e06da8 tempest-VolumeQuotaClassesTest-958998017 tempest-VolumeQuotaClassesTest-958998017-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:29:58.755727 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f96f23-5663-485e-81bb-690750e06da8 tempest-VolumeQuotaClassesTest-958998017 tempest-VolumeQuotaClassesTest-958998017-project-admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:29:58.756291 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f96f23-5663-485e-81bb-690750e06da8 tempest-VolumeQuotaClassesTest-958998017 tempest-VolumeQuotaClassesTest-958998017-project-admin] PATH_INFO: `/v3/projects/3b4736ba61a44ed295d866abd64e0712` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:29:58.758204 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f96f23-5663-485e-81bb-690750e06da8 tempest-VolumeQuotaClassesTest-958998017 tempest-VolumeQuotaClassesTest-958998017-project-admin] RBAC: Authorizing `identity:delete_project(project_id=3b4736ba61a44ed295d866abd64e0712)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:29:58.760687 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f96f23-5663-485e-81bb-690750e06da8 tempest-VolumeQuotaClassesTest-958998017 tempest-VolumeQuotaClassesTest-958998017-project-admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:29:58.782721 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfc3907-6aae-44b3-9d10-ca9f4d83f33c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:29:58.783337 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfc3907-6aae-44b3-9d10-ca9f4d83f33c admin admin] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:29:58.783620 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfc3907-6aae-44b3-9d10-ca9f4d83f33c admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:29:58.783897 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfc3907-6aae-44b3-9d10-ca9f4d83f33c admin admin] PATH_INFO: `/v3/users` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:29:58.784547 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfc3907-6aae-44b3-9d10-ca9f4d83f33c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:29:58.786626 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfc3907-6aae-44b3-9d10-ca9f4d83f33c admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:29:58.858996 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=97,keystone:INSERT=1,keystone:DELETE=2 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:29:58.861919 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1285/2579] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 01:29:58 2022] POST /identity/v3/users => generated 301 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 01:29:58.867241 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfc3907-6aae-44b3-9d10-ca9f4d83f33c admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:29:58.874475 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-b9f96f23-5663-485e-81bb-690750e06da8 tempest-VolumeQuotaClassesTest-958998017 tempest-VolumeQuotaClassesTest-958998017-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3b4736ba61a44ed295d866abd64e0712', 'request_id': 'req-b9f96f23-5663-485e-81bb-690750e06da8'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:29:58.876078 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfc3907-6aae-44b3-9d10-ca9f4d83f33c admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:29:58.880290 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-b9f96f23-5663-485e-81bb-690750e06da8 tempest-VolumeQuotaClassesTest-958998017 tempest-VolumeQuotaClassesTest-958998017-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3b4736ba61a44ed295d866abd64e0712', 'request_id': 'req-b9f96f23-5663-485e-81bb-690750e06da8'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:29:58.896806 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:29:58.938785 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc2829c-0ba5-4b91-815c-e4f4aaf44388 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:29:58.939266 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc2829c-0ba5-4b91-815c-e4f4aaf44388 admin admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:29:58.939543 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc2829c-0ba5-4b91-815c-e4f4aaf44388 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:29:58.939834 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc2829c-0ba5-4b91-815c-e4f4aaf44388 admin admin] PATH_INFO: `/v3/roles` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:29:58.940531 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc2829c-0ba5-4b91-815c-e4f4aaf44388 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:29:58.941662 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-b9f96f23-5663-485e-81bb-690750e06da8 tempest-VolumeQuotaClassesTest-958998017 tempest-VolumeQuotaClassesTest-958998017-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3b4736ba61a44ed295d866abd64e0712', 'request_id': 'req-b9f96f23-5663-485e-81bb-690750e06da8'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:29:58.942169 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc2829c-0ba5-4b91-815c-e4f4aaf44388 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:29:58.952155 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1286/2580] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:29:58 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 01:29:58.954045 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-b9f96f23-5663-485e-81bb-690750e06da8 tempest-VolumeQuotaClassesTest-958998017 tempest-VolumeQuotaClassesTest-958998017-project-admin] The token cache is being invalidate because project 3b4736ba61a44ed295d866abd64e0712 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81750) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 01:29:58.954344 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-b9f96f23-5663-485e-81bb-690750e06da8 tempest-VolumeQuotaClassesTest-958998017 tempest-VolumeQuotaClassesTest-958998017-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:29:58.955278 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-b9f96f23-5663-485e-81bb-690750e06da8 tempest-VolumeQuotaClassesTest-958998017 tempest-VolumeQuotaClassesTest-958998017-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:29:58.956517 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc2829c-0ba5-4b91-815c-e4f4aaf44388 admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:29:58.956838 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1295/2581] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 01:29:58 2022] DELETE /identity/v3/projects/3b4736ba61a44ed295d866abd64e0712 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 01:29:58.958261 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-ccc2829c-0ba5-4b91-815c-e4f4aaf44388 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:29:58.971718 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc2829c-0ba5-4b91-815c-e4f4aaf44388 admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:29:58.982402 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38b76ae-86b4-49c3-b3c5-82478a874414 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:29:58.983125 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38b76ae-86b4-49c3-b3c5-82478a874414 admin admin] REQUEST_METHOD: `PUT` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:29:58.983416 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38b76ae-86b4-49c3-b3c5-82478a874414 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:29:58.983766 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38b76ae-86b4-49c3-b3c5-82478a874414 admin admin] PATH_INFO: `/v3/projects/7dbd312942204556bc410152c2ff84ce/users/4bdc5a4df6bf4da687d834b0662778a2/roles/d009411496b8452b88e77ffb8db015a7` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:29:59.001225 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38b76ae-86b4-49c3-b3c5-82478a874414 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7dbd312942204556bc410152c2ff84ce, user_id=4bdc5a4df6bf4da687d834b0662778a2, role_id=d009411496b8452b88e77ffb8db015a7)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:29:59.004094 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38b76ae-86b4-49c3-b3c5-82478a874414 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:29:59.093492 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1287/2582] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 01:29:58 2022] PUT /identity/v3/projects/7dbd312942204556bc410152c2ff84ce/users/4bdc5a4df6bf4da687d834b0662778a2/roles/d009411496b8452b88e77ffb8db015a7 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 01:29:59.101724 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f96f23-5663-485e-81bb-690750e06da8 tempest-VolumeQuotaClassesTest-958998017 tempest-VolumeQuotaClassesTest-958998017-project-admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:29:59.112556 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f96f23-5663-485e-81bb-690750e06da8 tempest-VolumeQuotaClassesTest-958998017 tempest-VolumeQuotaClassesTest-958998017-project-admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:29:59.167617 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c4142c-9813-4867-8d6d-6e2300764e55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:29:59.168242 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c4142c-9813-4867-8d6d-6e2300764e55 admin admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:29:59.168529 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c4142c-9813-4867-8d6d-6e2300764e55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:29:59.168842 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c4142c-9813-4867-8d6d-6e2300764e55 admin admin] PATH_INFO: `/v3/roles` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:29:59.169480 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c4142c-9813-4867-8d6d-6e2300764e55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:29:59.171152 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c4142c-9813-4867-8d6d-6e2300764e55 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:29:59.180759 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1296/2583] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:29:59 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:29:59.184993 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38b76ae-86b4-49c3-b3c5-82478a874414 admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:29:59.195001 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38b76ae-86b4-49c3-b3c5-82478a874414 admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:29:59.200096 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f80d6fb-7c39-43eb-aebd-7ab521f2e95f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:29:59.200750 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f80d6fb-7c39-43eb-aebd-7ab521f2e95f admin admin] REQUEST_METHOD: `PUT` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:29:59.201040 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f80d6fb-7c39-43eb-aebd-7ab521f2e95f admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:29:59.201316 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f80d6fb-7c39-43eb-aebd-7ab521f2e95f admin admin] PATH_INFO: `/v3/projects/7dbd312942204556bc410152c2ff84ce/users/4bdc5a4df6bf4da687d834b0662778a2/roles/d009411496b8452b88e77ffb8db015a7` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:29:59.203205 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f80d6fb-7c39-43eb-aebd-7ab521f2e95f admin admin] RBAC: Authorizing `identity:create_grant(project_id=7dbd312942204556bc410152c2ff84ce, user_id=4bdc5a4df6bf4da687d834b0662778a2, role_id=d009411496b8452b88e77ffb8db015a7)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:29:59.206130 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f80d6fb-7c39-43eb-aebd-7ab521f2e95f admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:29:59.216806 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1288/2584] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 01:29:59 2022] PUT /identity/v3/projects/7dbd312942204556bc410152c2ff84ce/users/4bdc5a4df6bf4da687d834b0662778a2/roles/d009411496b8452b88e77ffb8db015a7 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 01:29:59.227198 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e512060-f117-490b-a1f5-9bef4d44d7db None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:29:59.227488 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e512060-f117-490b-a1f5-9bef4d44d7db None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:29:59.227767 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e512060-f117-490b-a1f5-9bef4d44d7db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:29:59.258151 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-3e512060-f117-490b-a1f5-9bef4d44d7db None None] MFA Rules not processed for user `4bdc5a4df6bf4da687d834b0662778a2`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:29:59.259890 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-3e512060-f117-490b-a1f5-9bef4d44d7db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:29:59.328203 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1297/2585] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:29:59 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:29:59.340124 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f80d6fb-7c39-43eb-aebd-7ab521f2e95f admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:29:59.341900 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-6f80d6fb-7c39-43eb-aebd-7ab521f2e95f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:29:59.399197 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f80d6fb-7c39-43eb-aebd-7ab521f2e95f admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:29:59.403489 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dafe05-72b6-4fb3-8021-cbf5d87a9870 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5c8fc69c518412ba1aad61a9f4f558c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:29:59.404091 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dafe05-72b6-4fb3-8021-cbf5d87a9870 service cinder] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:29:59.404427 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dafe05-72b6-4fb3-8021-cbf5d87a9870 service cinder] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:29:59.404705 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dafe05-72b6-4fb3-8021-cbf5d87a9870 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:29:59.413920 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13dafe05-72b6-4fb3-8021-cbf5d87a9870 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:29:59.421431 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13dafe05-72b6-4fb3-8021-cbf5d87a9870 service cinder] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:29:59.424738 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1289/2586] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:29:59 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:29:59.566268 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e512060-f117-490b-a1f5-9bef4d44d7db None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:29:59.608696 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:29:59.629501 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e512060-f117-490b-a1f5-9bef4d44d7db None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:29:59.634888 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e03ad7-22fe-46a3-a261-178c99ceddcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:29:59.635557 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e03ad7-22fe-46a3-a261-178c99ceddcf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:29:59.635841 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e03ad7-22fe-46a3-a261-178c99ceddcf admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:29:59.636118 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e03ad7-22fe-46a3-a261-178c99ceddcf admin admin] PATH_INFO: `/v3/users/d3dbce6124d24f739b61be17a5aeeb1f` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:29:59.637398 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e03ad7-22fe-46a3-a261-178c99ceddcf admin admin] RBAC: Authorizing `identity:delete_user(user_id=d3dbce6124d24f739b61be17a5aeeb1f)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:29:59.639511 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e03ad7-22fe-46a3-a261-178c99ceddcf admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:29:59.682718 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dafe05-72b6-4fb3-8021-cbf5d87a9870 service cinder] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:29:59.684125 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-13dafe05-72b6-4fb3-8021-cbf5d87a9870 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:29:59.697971 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dafe05-72b6-4fb3-8021-cbf5d87a9870 service cinder] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:29:59.702519 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff78193-42bd-4e42-bedf-74ec49cbfa9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:29:59.703130 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff78193-42bd-4e42-bedf-74ec49cbfa9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:29:59.703408 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff78193-42bd-4e42-bedf-74ec49cbfa9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:29:59.703677 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff78193-42bd-4e42-bedf-74ec49cbfa9a admin admin] PATH_INFO: `/v3/users/f42e604eb8fd43618e7f388cfeb4efca` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:29:59.704795 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff78193-42bd-4e42-bedf-74ec49cbfa9a admin admin] RBAC: Authorizing `identity:delete_user(user_id=f42e604eb8fd43618e7f388cfeb4efca)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:29:59.707011 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff78193-42bd-4e42-bedf-74ec49cbfa9a admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:29:59.823616 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-77e03ad7-22fe-46a3-a261-178c99ceddcf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd3dbce6124d24f739b61be17a5aeeb1f'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:29:59.835316 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-77e03ad7-22fe-46a3-a261-178c99ceddcf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd3dbce6124d24f739b61be17a5aeeb1f'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:29:59.835979 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-77e03ad7-22fe-46a3-a261-178c99ceddcf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd3dbce6124d24f739b61be17a5aeeb1f'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:29:59.842756 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-77e03ad7-22fe-46a3-a261-178c99ceddcf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd3dbce6124d24f739b61be17a5aeeb1f'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:29:59.875499 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-7ff78193-42bd-4e42-bedf-74ec49cbfa9a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f42e604eb8fd43618e7f388cfeb4efca'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:29:59.947158 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-77e03ad7-22fe-46a3-a261-178c99ceddcf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd3dbce6124d24f739b61be17a5aeeb1f'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:29:59.953630 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1298/2587] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 01:29:59 2022] DELETE /identity/v3/users/d3dbce6124d24f739b61be17a5aeeb1f => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:29:59.958186 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f21fd60-d2b6-4ea2-aa15-eadfa6262c51 None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:29:59.958470 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f21fd60-d2b6-4ea2-aa15-eadfa6262c51 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:29:59.958742 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f21fd60-d2b6-4ea2-aa15-eadfa6262c51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:29:59.977414 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-1f21fd60-d2b6-4ea2-aa15-eadfa6262c51 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:29:59.978624 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-1f21fd60-d2b6-4ea2-aa15-eadfa6262c51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:30:00.010173 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-7ff78193-42bd-4e42-bedf-74ec49cbfa9a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f42e604eb8fd43618e7f388cfeb4efca'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:00.010805 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-7ff78193-42bd-4e42-bedf-74ec49cbfa9a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f42e604eb8fd43618e7f388cfeb4efca'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:00.018472 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-7ff78193-42bd-4e42-bedf-74ec49cbfa9a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f42e604eb8fd43618e7f388cfeb4efca'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:00.019149 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-7ff78193-42bd-4e42-bedf-74ec49cbfa9a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f42e604eb8fd43618e7f388cfeb4efca'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:00.032654 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:30:00.037724 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1290/2588] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 01:29:59 2022] DELETE /identity/v3/users/f42e604eb8fd43618e7f388cfeb4efca => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:30:00.039284 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1299/2589] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:29:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 01:30:00.042544 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb7526a-01ea-41c9-9bbc-3d5fbc3fd93f None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:00.042890 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb7526a-01ea-41c9-9bbc-3d5fbc3fd93f None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:00.043202 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb7526a-01ea-41c9-9bbc-3d5fbc3fd93f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:00.050043 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f21fd60-d2b6-4ea2-aa15-eadfa6262c51 None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:30:00.051688 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-1f21fd60-d2b6-4ea2-aa15-eadfa6262c51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:30:00.064222 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-bbb7526a-01ea-41c9-9bbc-3d5fbc3fd93f None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:30:00.065562 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-bbb7526a-01ea-41c9-9bbc-3d5fbc3fd93f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:30:00.113097 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f21fd60-d2b6-4ea2-aa15-eadfa6262c51 None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:30:00.123778 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:30:00.132075 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1291/2590] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:30:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 01:30:00.143216 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb7526a-01ea-41c9-9bbc-3d5fbc3fd93f None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:30:00.172828 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33178c5-ef3e-413b-9d3b-6e54f242919e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0817c3320d5d41d8bba0550c8ae8dd60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:30:00.173522 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33178c5-ef3e-413b-9d3b-6e54f242919e service neutron] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:00.173954 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33178c5-ef3e-413b-9d3b-6e54f242919e service neutron] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:00.174302 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33178c5-ef3e-413b-9d3b-6e54f242919e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:00.184752 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33178c5-ef3e-413b-9d3b-6e54f242919e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:30:00.186393 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33178c5-ef3e-413b-9d3b-6e54f242919e service neutron] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:30:00.200774 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb7526a-01ea-41c9-9bbc-3d5fbc3fd93f None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:30:00.202705 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1300/2591] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:30:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 01:30:00.205542 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617e6eb9-8cbd-47a1-ab26-6f26fd7303df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0817c3320d5d41d8bba0550c8ae8dd60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:30:00.206159 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617e6eb9-8cbd-47a1-ab26-6f26fd7303df service neutron] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:00.208866 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617e6eb9-8cbd-47a1-ab26-6f26fd7303df service neutron] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:00.208866 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617e6eb9-8cbd-47a1-ab26-6f26fd7303df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:00.217506 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617e6eb9-8cbd-47a1-ab26-6f26fd7303df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:30:00.219046 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617e6eb9-8cbd-47a1-ab26-6f26fd7303df service neutron] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:30:00.228591 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1292/2592] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:30:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:30:00.720579 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33178c5-ef3e-413b-9d3b-6e54f242919e service neutron] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:30:00.723488 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-a33178c5-ef3e-413b-9d3b-6e54f242919e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:30:00.723643 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617e6eb9-8cbd-47a1-ab26-6f26fd7303df service neutron] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:30:00.724860 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-617e6eb9-8cbd-47a1-ab26-6f26fd7303df service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:30:00.736409 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33178c5-ef3e-413b-9d3b-6e54f242919e service neutron] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:30:00.742860 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacdd7b9-7084-4171-abc1-243b062f60d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:30:00.743502 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacdd7b9-7084-4171-abc1-243b062f60d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:00.743954 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacdd7b9-7084-4171-abc1-243b062f60d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:00.749367 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacdd7b9-7084-4171-abc1-243b062f60d2 admin admin] PATH_INFO: `/v3/projects/f52e1b886f364391942978ef33782d97` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:00.749367 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cacdd7b9-7084-4171-abc1-243b062f60d2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f52e1b886f364391942978ef33782d97)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:30:00.749367 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cacdd7b9-7084-4171-abc1-243b062f60d2 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:30:00.749687 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617e6eb9-8cbd-47a1-ab26-6f26fd7303df service neutron] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:30:00.754474 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51dccae-da52-40c3-8bd6-ab30cc848b6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:30:00.755153 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51dccae-da52-40c3-8bd6-ab30cc848b6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:00.755449 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51dccae-da52-40c3-8bd6-ab30cc848b6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:00.755720 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51dccae-da52-40c3-8bd6-ab30cc848b6a admin admin] PATH_INFO: `/v3/projects/10d98f3c49c54c9c8080b6d2ba96d6a2` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:00.757084 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51dccae-da52-40c3-8bd6-ab30cc848b6a admin admin] RBAC: Authorizing `identity:delete_project(project_id=10d98f3c49c54c9c8080b6d2ba96d6a2)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:30:00.760118 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51dccae-da52-40c3-8bd6-ab30cc848b6a admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:30:00.890960 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-cacdd7b9-7084-4171-abc1-243b062f60d2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f52e1b886f364391942978ef33782d97', 'request_id': 'req-cacdd7b9-7084-4171-abc1-243b062f60d2'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:00.896646 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-cacdd7b9-7084-4171-abc1-243b062f60d2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f52e1b886f364391942978ef33782d97', 'request_id': 'req-cacdd7b9-7084-4171-abc1-243b062f60d2'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:00.930029 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-e51dccae-da52-40c3-8bd6-ab30cc848b6a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '10d98f3c49c54c9c8080b6d2ba96d6a2', 'request_id': 'req-e51dccae-da52-40c3-8bd6-ab30cc848b6a'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:00.936229 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-e51dccae-da52-40c3-8bd6-ab30cc848b6a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '10d98f3c49c54c9c8080b6d2ba96d6a2', 'request_id': 'req-e51dccae-da52-40c3-8bd6-ab30cc848b6a'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:00.942546 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-e51dccae-da52-40c3-8bd6-ab30cc848b6a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '10d98f3c49c54c9c8080b6d2ba96d6a2', 'request_id': 'req-e51dccae-da52-40c3-8bd6-ab30cc848b6a'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:00.981403 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-cacdd7b9-7084-4171-abc1-243b062f60d2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f52e1b886f364391942978ef33782d97', 'request_id': 'req-cacdd7b9-7084-4171-abc1-243b062f60d2'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:00.993039 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-cacdd7b9-7084-4171-abc1-243b062f60d2 admin admin] The token cache is being invalidate because project f52e1b886f364391942978ef33782d97 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81750) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 01:30:00.993333 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-cacdd7b9-7084-4171-abc1-243b062f60d2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:00.993969 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-cacdd7b9-7084-4171-abc1-243b062f60d2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:00.995990 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1301/2593] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 01:30:00 2022] DELETE /identity/v3/projects/f52e1b886f364391942978ef33782d97 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:30:00.999930 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacdd7b9-7084-4171-abc1-243b062f60d2 admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:30:01.001332 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-cacdd7b9-7084-4171-abc1-243b062f60d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:30:01.055424 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-e51dccae-da52-40c3-8bd6-ab30cc848b6a admin admin] The token cache is being invalidate because project 10d98f3c49c54c9c8080b6d2ba96d6a2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81749) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 01:30:01.055777 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-e51dccae-da52-40c3-8bd6-ab30cc848b6a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:01.056395 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-e51dccae-da52-40c3-8bd6-ab30cc848b6a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:01.058392 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1293/2594] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 01:30:00 2022] DELETE /identity/v3/projects/10d98f3c49c54c9c8080b6d2ba96d6a2 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:30:01.062404 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51dccae-da52-40c3-8bd6-ab30cc848b6a admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:30:01.063693 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-e51dccae-da52-40c3-8bd6-ab30cc848b6a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:30:01.063969 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacdd7b9-7084-4171-abc1-243b062f60d2 admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:30:01.091917 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:30:01.109226 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:30:01.122256 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9637d66f-abd5-4db6-9611-7d7bce21a0ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:30:01.122898 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9637d66f-abd5-4db6-9611-7d7bce21a0ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:01.123179 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9637d66f-abd5-4db6-9611-7d7bce21a0ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:01.123452 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9637d66f-abd5-4db6-9611-7d7bce21a0ab admin admin] PATH_INFO: `/v3/users/4bdc5a4df6bf4da687d834b0662778a2` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:01.124583 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9637d66f-abd5-4db6-9611-7d7bce21a0ab admin admin] RBAC: Authorizing `identity:delete_user(user_id=4bdc5a4df6bf4da687d834b0662778a2)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:30:01.126622 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9637d66f-abd5-4db6-9611-7d7bce21a0ab admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:30:01.128629 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51dccae-da52-40c3-8bd6-ab30cc848b6a admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:30:01.133275 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc160bf-66ca-4080-bae9-9ce02f8f054f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:30:01.133901 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc160bf-66ca-4080-bae9-9ce02f8f054f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:01.134187 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc160bf-66ca-4080-bae9-9ce02f8f054f admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:01.134463 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc160bf-66ca-4080-bae9-9ce02f8f054f admin admin] PATH_INFO: `/v3/users/677cf03698bd424282e7a043d33ed921` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:01.136342 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc160bf-66ca-4080-bae9-9ce02f8f054f admin admin] RBAC: Authorizing `identity:delete_user(user_id=677cf03698bd424282e7a043d33ed921)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:30:01.138312 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc160bf-66ca-4080-bae9-9ce02f8f054f admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:30:01.292545 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-9637d66f-abd5-4db6-9611-7d7bce21a0ab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4bdc5a4df6bf4da687d834b0662778a2'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:01.304250 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-9637d66f-abd5-4db6-9611-7d7bce21a0ab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4bdc5a4df6bf4da687d834b0662778a2'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:01.304973 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-9637d66f-abd5-4db6-9611-7d7bce21a0ab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4bdc5a4df6bf4da687d834b0662778a2'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:01.315523 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-9637d66f-abd5-4db6-9611-7d7bce21a0ab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4bdc5a4df6bf4da687d834b0662778a2'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:01.326852 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-dfc160bf-66ca-4080-bae9-9ce02f8f054f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '677cf03698bd424282e7a043d33ed921'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:01.398670 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-9637d66f-abd5-4db6-9611-7d7bce21a0ab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4bdc5a4df6bf4da687d834b0662778a2'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:01.405479 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1302/2595] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 01:30:00 2022] DELETE /identity/v3/users/4bdc5a4df6bf4da687d834b0662778a2 => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 01:30:01.461597 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-dfc160bf-66ca-4080-bae9-9ce02f8f054f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '677cf03698bd424282e7a043d33ed921'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:01.462246 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-dfc160bf-66ca-4080-bae9-9ce02f8f054f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '677cf03698bd424282e7a043d33ed921'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:01.469886 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-dfc160bf-66ca-4080-bae9-9ce02f8f054f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '677cf03698bd424282e7a043d33ed921'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:01.470553 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-dfc160bf-66ca-4080-bae9-9ce02f8f054f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '677cf03698bd424282e7a043d33ed921'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:01.489745 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1294/2596] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 01:30:01 2022] DELETE /identity/v3/users/677cf03698bd424282e7a043d33ed921 => generated 0 bytes in 427 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 01:30:01.914545 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9637d66f-abd5-4db6-9611-7d7bce21a0ab admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:30:01.916057 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-9637d66f-abd5-4db6-9611-7d7bce21a0ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:30:01.945011 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc160bf-66ca-4080-bae9-9ce02f8f054f admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:30:01.946534 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-dfc160bf-66ca-4080-bae9-9ce02f8f054f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:30:01.998174 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9637d66f-abd5-4db6-9611-7d7bce21a0ab admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:30:02.015806 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc160bf-66ca-4080-bae9-9ce02f8f054f admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:30:02.020602 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68221d8-3d63-4d9c-b8b6-a700da7b7550 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:30:02.022192 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68221d8-3d63-4d9c-b8b6-a700da7b7550 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:02.022192 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68221d8-3d63-4d9c-b8b6-a700da7b7550 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:02.022192 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68221d8-3d63-4d9c-b8b6-a700da7b7550 admin admin] PATH_INFO: `/v3/projects/5f0b244db4db41baba47aefc52cb4c7b` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:02.023019 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68221d8-3d63-4d9c-b8b6-a700da7b7550 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5f0b244db4db41baba47aefc52cb4c7b)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:30:02.025073 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68221d8-3d63-4d9c-b8b6-a700da7b7550 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:30:02.058258 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2aec4ce-3b6a-49df-adba-7802302fd60d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:30:02.058931 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2aec4ce-3b6a-49df-adba-7802302fd60d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:02.059215 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2aec4ce-3b6a-49df-adba-7802302fd60d admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:02.059494 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2aec4ce-3b6a-49df-adba-7802302fd60d admin admin] PATH_INFO: `/v3/projects/7dbd312942204556bc410152c2ff84ce` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:02.060663 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2aec4ce-3b6a-49df-adba-7802302fd60d admin admin] RBAC: Authorizing `identity:delete_project(project_id=7dbd312942204556bc410152c2ff84ce)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:30:02.062630 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2aec4ce-3b6a-49df-adba-7802302fd60d admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:30:02.078054 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:30:02.122775 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:30:02.125726 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-a68221d8-3d63-4d9c-b8b6-a700da7b7550 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5f0b244db4db41baba47aefc52cb4c7b', 'request_id': 'req-a68221d8-3d63-4d9c-b8b6-a700da7b7550'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:02.131114 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-a68221d8-3d63-4d9c-b8b6-a700da7b7550 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5f0b244db4db41baba47aefc52cb4c7b', 'request_id': 'req-a68221d8-3d63-4d9c-b8b6-a700da7b7550'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:02.137367 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-a68221d8-3d63-4d9c-b8b6-a700da7b7550 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5f0b244db4db41baba47aefc52cb4c7b', 'request_id': 'req-a68221d8-3d63-4d9c-b8b6-a700da7b7550'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:02.164271 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-b2aec4ce-3b6a-49df-adba-7802302fd60d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7dbd312942204556bc410152c2ff84ce', 'request_id': 'req-b2aec4ce-3b6a-49df-adba-7802302fd60d'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:02.170224 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-b2aec4ce-3b6a-49df-adba-7802302fd60d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7dbd312942204556bc410152c2ff84ce', 'request_id': 'req-b2aec4ce-3b6a-49df-adba-7802302fd60d'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:02.226437 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-a68221d8-3d63-4d9c-b8b6-a700da7b7550 admin admin] The token cache is being invalidate because project 5f0b244db4db41baba47aefc52cb4c7b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81749) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 01:30:02.226759 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-a68221d8-3d63-4d9c-b8b6-a700da7b7550 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:02.227384 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-a68221d8-3d63-4d9c-b8b6-a700da7b7550 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:02.228931 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1295/2597] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 01:30:01 2022] DELETE /identity/v3/projects/5f0b244db4db41baba47aefc52cb4c7b => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:30:02.233658 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6571e63d-d5c2-4f78-aaa9-25554f2bedb1 None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:02.233950 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6571e63d-d5c2-4f78-aaa9-25554f2bedb1 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:02.234228 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6571e63d-d5c2-4f78-aaa9-25554f2bedb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:02.253159 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-6571e63d-d5c2-4f78-aaa9-25554f2bedb1 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:30:02.254485 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-6571e63d-d5c2-4f78-aaa9-25554f2bedb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:30:02.267098 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-b2aec4ce-3b6a-49df-adba-7802302fd60d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7dbd312942204556bc410152c2ff84ce', 'request_id': 'req-b2aec4ce-3b6a-49df-adba-7802302fd60d'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:02.279590 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-b2aec4ce-3b6a-49df-adba-7802302fd60d admin admin] The token cache is being invalidate because project 7dbd312942204556bc410152c2ff84ce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81750) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 01:30:02.279885 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-b2aec4ce-3b6a-49df-adba-7802302fd60d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:02.280524 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-b2aec4ce-3b6a-49df-adba-7802302fd60d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:02.282267 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1303/2598] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 01:30:01 2022] DELETE /identity/v3/projects/7dbd312942204556bc410152c2ff84ce => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:30:02.287344 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284443a7-3bcf-4786-9d0b-01f6c43b9a51 None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:02.287635 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284443a7-3bcf-4786-9d0b-01f6c43b9a51 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:02.287916 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284443a7-3bcf-4786-9d0b-01f6c43b9a51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:02.307095 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-284443a7-3bcf-4786-9d0b-01f6c43b9a51 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:30:02.308356 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-284443a7-3bcf-4786-9d0b-01f6c43b9a51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:30:02.317563 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1296/2599] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:30:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 01:30:02.325699 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79729da7-ac46-42e2-8da6-d94a38ea791f None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:02.325994 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79729da7-ac46-42e2-8da6-d94a38ea791f None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:02.326278 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79729da7-ac46-42e2-8da6-d94a38ea791f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:02.347083 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-79729da7-ac46-42e2-8da6-d94a38ea791f None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:30:02.348299 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-79729da7-ac46-42e2-8da6-d94a38ea791f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:30:02.357378 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1297/2600] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:30:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 01:30:02.361315 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79729da7-ac46-42e2-8da6-d94a38ea791f None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:30:02.368249 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1304/2601] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:30:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 01:30:02.370749 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79729da7-ac46-42e2-8da6-d94a38ea791f None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:30:02.375565 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64895114-30a8-4a8f-b4ee-ede9a217cb6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:30:02.376030 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685b2f72-8b29-4eff-90fc-efb0ebf44679 None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:02.376173 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64895114-30a8-4a8f-b4ee-ede9a217cb6c admin admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:02.376320 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685b2f72-8b29-4eff-90fc-efb0ebf44679 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:02.376419 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64895114-30a8-4a8f-b4ee-ede9a217cb6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:02.376591 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685b2f72-8b29-4eff-90fc-efb0ebf44679 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:02.376686 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64895114-30a8-4a8f-b4ee-ede9a217cb6c admin admin] PATH_INFO: `/v3/domains` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:02.377364 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64895114-30a8-4a8f-b4ee-ede9a217cb6c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:30:02.377673 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64895114-30a8-4a8f-b4ee-ede9a217cb6c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:30:02.379314 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64895114-30a8-4a8f-b4ee-ede9a217cb6c admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:30:02.391786 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1298/2602] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Aug 31 01:30:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 01:30:02.396077 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64895114-30a8-4a8f-b4ee-ede9a217cb6c admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:30:02.397203 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-685b2f72-8b29-4eff-90fc-efb0ebf44679 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:30:02.398428 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-685b2f72-8b29-4eff-90fc-efb0ebf44679 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:30:02.407084 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64895114-30a8-4a8f-b4ee-ede9a217cb6c admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:30:02.407468 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1305/2603] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:30:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 01:30:02.412006 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca0d5f6-a3ac-474d-8cb1-83fdfd903db6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:30:02.412603 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca0d5f6-a3ac-474d-8cb1-83fdfd903db6 admin admin] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:02.412916 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca0d5f6-a3ac-474d-8cb1-83fdfd903db6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:02.413130 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685b2f72-8b29-4eff-90fc-efb0ebf44679 None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:30:02.413261 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca0d5f6-a3ac-474d-8cb1-83fdfd903db6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:02.413802 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca0d5f6-a3ac-474d-8cb1-83fdfd903db6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:30:02.415741 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca0d5f6-a3ac-474d-8cb1-83fdfd903db6 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:30:02.422792 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685b2f72-8b29-4eff-90fc-efb0ebf44679 None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:30:02.427378 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524a9da5-d155-4487-82fe-d7472976e62c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:30:02.428027 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524a9da5-d155-4487-82fe-d7472976e62c admin admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:02.428310 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524a9da5-d155-4487-82fe-d7472976e62c admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:02.428586 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524a9da5-d155-4487-82fe-d7472976e62c admin admin] PATH_INFO: `/v3/domains` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:02.429249 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524a9da5-d155-4487-82fe-d7472976e62c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:30:02.429554 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524a9da5-d155-4487-82fe-d7472976e62c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:30:02.431187 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524a9da5-d155-4487-82fe-d7472976e62c admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:30:02.443486 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1306/2604] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Aug 31 01:30:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 01:30:02.447811 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524a9da5-d155-4487-82fe-d7472976e62c admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:30:02.457640 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524a9da5-d155-4487-82fe-d7472976e62c admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:30:02.462292 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd362eef-853e-4a09-8f4c-1b1b442fa5e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:30:02.462872 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd362eef-853e-4a09-8f4c-1b1b442fa5e3 admin admin] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:02.463156 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd362eef-853e-4a09-8f4c-1b1b442fa5e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:02.463976 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd362eef-853e-4a09-8f4c-1b1b442fa5e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:02.464597 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd362eef-853e-4a09-8f4c-1b1b442fa5e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:30:02.466477 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd362eef-853e-4a09-8f4c-1b1b442fa5e3 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:30:02.480010 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1299/2605] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 01:30:02 2022] POST /identity/v3/projects => generated 407 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 01:30:02.484303 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca0d5f6-a3ac-474d-8cb1-83fdfd903db6 admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:30:02.545297 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca0d5f6-a3ac-474d-8cb1-83fdfd903db6 admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:30:02.550833 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3e55d2-4507-448f-a826-4eaba3ca4f26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:30:02.551450 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3e55d2-4507-448f-a826-4eaba3ca4f26 admin admin] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:02.552077 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3e55d2-4507-448f-a826-4eaba3ca4f26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:02.555081 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3e55d2-4507-448f-a826-4eaba3ca4f26 admin admin] PATH_INFO: `/v3/users` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:02.555081 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3e55d2-4507-448f-a826-4eaba3ca4f26 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:30:02.555428 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3e55d2-4507-448f-a826-4eaba3ca4f26 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:30:02.564275 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1307/2606] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 01:30:02 2022] POST /identity/v3/projects => generated 373 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 01:30:02.578762 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd362eef-853e-4a09-8f4c-1b1b442fa5e3 admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:30:02.628218 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd362eef-853e-4a09-8f4c-1b1b442fa5e3 admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:30:02.632833 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc678420-43d7-4ba9-b595-d7ed70ab5f99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:30:02.633432 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc678420-43d7-4ba9-b595-d7ed70ab5f99 admin admin] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:02.633714 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc678420-43d7-4ba9-b595-d7ed70ab5f99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:02.633987 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc678420-43d7-4ba9-b595-d7ed70ab5f99 admin admin] PATH_INFO: `/v3/users` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:02.634601 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc678420-43d7-4ba9-b595-d7ed70ab5f99 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:30:02.636561 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc678420-43d7-4ba9-b595-d7ed70ab5f99 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:30:02.645642 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:30:02.681400 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1300/2607] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 01:30:02 2022] POST /identity/v3/users => generated 304 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:30:02.685128 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3e55d2-4507-448f-a826-4eaba3ca4f26 admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:30:02.697242 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3e55d2-4507-448f-a826-4eaba3ca4f26 admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:30:02.702890 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1308/2608] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 01:30:02 2022] POST /identity/v3/users => generated 287 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:30:02.704242 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4749b071-890e-4415-8cdd-8bd9ec806f7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:30:02.704909 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4749b071-890e-4415-8cdd-8bd9ec806f7f admin admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:02.705569 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4749b071-890e-4415-8cdd-8bd9ec806f7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:02.705856 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4749b071-890e-4415-8cdd-8bd9ec806f7f admin admin] PATH_INFO: `/v3/roles` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:02.706515 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4749b071-890e-4415-8cdd-8bd9ec806f7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:30:02.708617 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4749b071-890e-4415-8cdd-8bd9ec806f7f admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:30:02.708945 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc678420-43d7-4ba9-b595-d7ed70ab5f99 admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:30:02.723107 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1301/2609] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:30:02 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 01:30:02.723464 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc678420-43d7-4ba9-b595-d7ed70ab5f99 admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:30:02.724181 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4749b071-890e-4415-8cdd-8bd9ec806f7f admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:30:02.728249 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9b6524-e9c0-465d-943e-f1cda6b85fc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:30:02.733352 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9b6524-e9c0-465d-943e-f1cda6b85fc1 admin admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:02.733352 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9b6524-e9c0-465d-943e-f1cda6b85fc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:02.733352 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9b6524-e9c0-465d-943e-f1cda6b85fc1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:02.733352 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9b6524-e9c0-465d-943e-f1cda6b85fc1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:30:02.733352 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9b6524-e9c0-465d-943e-f1cda6b85fc1 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:30:02.734908 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4749b071-890e-4415-8cdd-8bd9ec806f7f admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:30:02.739790 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383d2bff-fb80-449d-8092-3241671cb91e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:30:02.740388 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383d2bff-fb80-449d-8092-3241671cb91e admin admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:02.740679 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383d2bff-fb80-449d-8092-3241671cb91e admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:02.741022 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383d2bff-fb80-449d-8092-3241671cb91e admin admin] PATH_INFO: `/v3/roles` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:02.741693 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383d2bff-fb80-449d-8092-3241671cb91e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:30:02.742205 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1309/2610] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:30:02 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:30:02.743323 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383d2bff-fb80-449d-8092-3241671cb91e admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:30:02.746049 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9b6524-e9c0-465d-943e-f1cda6b85fc1 admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:30:02.748901 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:DELETE=4,keystone:SELECT=91,keystone:INSERT=5 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:30:02.755668 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9b6524-e9c0-465d-943e-f1cda6b85fc1 admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:30:02.760233 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1302/2611] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:30:02 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:30:02.760980 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03c17a7-af62-4856-acef-b1096aab30c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:30:02.761440 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03c17a7-af62-4856-acef-b1096aab30c0 admin admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:02.761719 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03c17a7-af62-4856-acef-b1096aab30c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:02.761994 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03c17a7-af62-4856-acef-b1096aab30c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:02.762633 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03c17a7-af62-4856-acef-b1096aab30c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:30:02.764026 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383d2bff-fb80-449d-8092-3241671cb91e admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:30:02.764214 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03c17a7-af62-4856-acef-b1096aab30c0 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:30:02.776099 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383d2bff-fb80-449d-8092-3241671cb91e admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:30:02.777961 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1310/2612] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:30:02 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 01:30:02.781579 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03c17a7-af62-4856-acef-b1096aab30c0 admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:30:02.785097 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7408d8b-5632-4846-8e0f-2e211687c59d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:30:02.785097 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7408d8b-5632-4846-8e0f-2e211687c59d admin admin] REQUEST_METHOD: `PUT` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:02.785097 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7408d8b-5632-4846-8e0f-2e211687c59d admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:02.785097 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7408d8b-5632-4846-8e0f-2e211687c59d admin admin] PATH_INFO: `/v3/projects/a3f745e0cc9c4f9b91aedb77410ca019/users/e1fdb3fa8ebd4e9eae78e1abb91842e7/roles/d0b9768f4d014766b9326387f2e11f34` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:02.790955 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03c17a7-af62-4856-acef-b1096aab30c0 admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:30:02.795613 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b197f2-ff5e-4e4f-9b69-2d6bc49b4751 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:30:02.796244 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b197f2-ff5e-4e4f-9b69-2d6bc49b4751 admin admin] REQUEST_METHOD: `PUT` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:02.796523 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b197f2-ff5e-4e4f-9b69-2d6bc49b4751 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:02.796821 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b197f2-ff5e-4e4f-9b69-2d6bc49b4751 admin admin] PATH_INFO: `/v3/projects/940c6050a62a42429fbf173565acffa5/users/0d129b0cc9604fc48a269b4465585ac3/roles/d0b9768f4d014766b9326387f2e11f34` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:02.798623 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7408d8b-5632-4846-8e0f-2e211687c59d admin admin] RBAC: Authorizing `identity:create_grant(project_id=a3f745e0cc9c4f9b91aedb77410ca019, user_id=e1fdb3fa8ebd4e9eae78e1abb91842e7, role_id=d0b9768f4d014766b9326387f2e11f34)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:30:02.801875 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7408d8b-5632-4846-8e0f-2e211687c59d admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:30:02.811579 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b197f2-ff5e-4e4f-9b69-2d6bc49b4751 admin admin] RBAC: Authorizing `identity:create_grant(project_id=940c6050a62a42429fbf173565acffa5, user_id=0d129b0cc9604fc48a269b4465585ac3, role_id=d0b9768f4d014766b9326387f2e11f34)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:30:02.814425 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b197f2-ff5e-4e4f-9b69-2d6bc49b4751 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:30:02.895699 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1303/2613] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 01:30:02 2022] PUT /identity/v3/projects/a3f745e0cc9c4f9b91aedb77410ca019/users/e1fdb3fa8ebd4e9eae78e1abb91842e7/roles/d0b9768f4d014766b9326387f2e11f34 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 01:30:02.906687 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f4e59b-7a15-4d2b-9a70-fcf100e373f1 None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:02.906975 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f4e59b-7a15-4d2b-9a70-fcf100e373f1 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:02.907247 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f4e59b-7a15-4d2b-9a70-fcf100e373f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:02.940677 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1311/2614] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 01:30:02 2022] PUT /identity/v3/projects/940c6050a62a42429fbf173565acffa5/users/0d129b0cc9604fc48a269b4465585ac3/roles/d0b9768f4d014766b9326387f2e11f34 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 01:30:02.941997 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-b4f4e59b-7a15-4d2b-9a70-fcf100e373f1 None None] MFA Rules not processed for user `e1fdb3fa8ebd4e9eae78e1abb91842e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:30:02.941997 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-b4f4e59b-7a15-4d2b-9a70-fcf100e373f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:30:02.942632 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8470bd-ac6d-4e03-9e0a-be152f621d2d None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:02.942926 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8470bd-ac6d-4e03-9e0a-be152f621d2d None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:02.943209 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8470bd-ac6d-4e03-9e0a-be152f621d2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:02.972023 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-1a8470bd-ac6d-4e03-9e0a-be152f621d2d None None] MFA Rules not processed for user `0d129b0cc9604fc48a269b4465585ac3`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:30:02.973381 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-1a8470bd-ac6d-4e03-9e0a-be152f621d2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:30:03.006847 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1304/2615] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:30:02 2022] POST /identity/v3/auth/tokens => generated 3284 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 01:30:03.016823 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f4e59b-7a15-4d2b-9a70-fcf100e373f1 None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:30:03.044828 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1312/2616] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:30:02 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 01:30:03.056262 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8470bd-ac6d-4e03-9e0a-be152f621d2d None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:30:03.057740 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-1a8470bd-ac6d-4e03-9e0a-be152f621d2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:30:03.079067 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f4e59b-7a15-4d2b-9a70-fcf100e373f1 None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:30:03.084039 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfeb91f4-1c43-4b20-843b-ce66b22a6bce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:30:03.084641 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfeb91f4-1c43-4b20-843b-ce66b22a6bce admin admin] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:03.085028 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfeb91f4-1c43-4b20-843b-ce66b22a6bce admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:03.085304 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfeb91f4-1c43-4b20-843b-ce66b22a6bce admin admin] PATH_INFO: `/v3/projects` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:03.085936 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfeb91f4-1c43-4b20-843b-ce66b22a6bce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:30:03.087828 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfeb91f4-1c43-4b20-843b-ce66b22a6bce admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:30:03.118015 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8470bd-ac6d-4e03-9e0a-be152f621d2d None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:30:03.121803 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995d3567-8c26-47b1-b86f-9c996b38bb7b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5c8fc69c518412ba1aad61a9f4f558c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:30:03.122462 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995d3567-8c26-47b1-b86f-9c996b38bb7b service cinder] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:03.122771 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995d3567-8c26-47b1-b86f-9c996b38bb7b service cinder] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:03.123124 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995d3567-8c26-47b1-b86f-9c996b38bb7b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:03.131403 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995d3567-8c26-47b1-b86f-9c996b38bb7b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:30:03.132888 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995d3567-8c26-47b1-b86f-9c996b38bb7b service cinder] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:30:03.142341 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1313/2617] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:30:03 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 01:30:03.163709 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995d3567-8c26-47b1-b86f-9c996b38bb7b service cinder] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:30:03.165380 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1305/2618] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 01:30:03 2022] POST /identity/v3/projects => generated 409 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 01:30:03.169582 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfeb91f4-1c43-4b20-843b-ce66b22a6bce admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:30:03.176879 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995d3567-8c26-47b1-b86f-9c996b38bb7b service cinder] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:30:03.213337 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:30:03.222530 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:30:03.234115 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfeb91f4-1c43-4b20-843b-ce66b22a6bce admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:30:03.236442 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2051ee3-253c-4b12-a5df-aca652a4cbb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:30:03.237130 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2051ee3-253c-4b12-a5df-aca652a4cbb1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:03.237684 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2051ee3-253c-4b12-a5df-aca652a4cbb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:03.238084 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2051ee3-253c-4b12-a5df-aca652a4cbb1 admin admin] PATH_INFO: `/v3/users/0d129b0cc9604fc48a269b4465585ac3` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:03.239190 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77558d90-114e-4ffe-ab8a-63cbb7440e26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:30:03.239598 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2051ee3-253c-4b12-a5df-aca652a4cbb1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d129b0cc9604fc48a269b4465585ac3)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:30:03.239855 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77558d90-114e-4ffe-ab8a-63cbb7440e26 admin admin] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:03.240149 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77558d90-114e-4ffe-ab8a-63cbb7440e26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:03.240431 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77558d90-114e-4ffe-ab8a-63cbb7440e26 admin admin] PATH_INFO: `/v3/users` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:03.241118 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77558d90-114e-4ffe-ab8a-63cbb7440e26 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:30:03.241615 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2051ee3-253c-4b12-a5df-aca652a4cbb1 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:30:03.243118 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77558d90-114e-4ffe-ab8a-63cbb7440e26 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:30:03.336659 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1306/2619] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 01:30:03 2022] POST /identity/v3/users => generated 311 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 01:30:03.340565 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77558d90-114e-4ffe-ab8a-63cbb7440e26 admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:30:03.350244 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77558d90-114e-4ffe-ab8a-63cbb7440e26 admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:30:03.355995 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673074e7-21fb-47ce-af8f-a2d0a7e5183d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:30:03.356588 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673074e7-21fb-47ce-af8f-a2d0a7e5183d admin admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:03.356969 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673074e7-21fb-47ce-af8f-a2d0a7e5183d admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:03.357326 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673074e7-21fb-47ce-af8f-a2d0a7e5183d admin admin] PATH_INFO: `/v3/roles` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:03.358181 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673074e7-21fb-47ce-af8f-a2d0a7e5183d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:30:03.360110 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673074e7-21fb-47ce-af8f-a2d0a7e5183d admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:30:03.372715 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1307/2620] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:30:03 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 01:30:03.376918 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673074e7-21fb-47ce-af8f-a2d0a7e5183d admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:30:03.387732 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673074e7-21fb-47ce-af8f-a2d0a7e5183d admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:30:03.392501 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6815c20-c811-4fe7-84bc-ab0385704673 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:30:03.393185 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6815c20-c811-4fe7-84bc-ab0385704673 admin admin] REQUEST_METHOD: `PUT` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:03.393479 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6815c20-c811-4fe7-84bc-ab0385704673 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:03.393754 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6815c20-c811-4fe7-84bc-ab0385704673 admin admin] PATH_INFO: `/v3/projects/80e8e30abf5c4629bb06bc5e24d23888/users/1888662e4b874d40b48594d9ea38a4d5/roles/d009411496b8452b88e77ffb8db015a7` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:03.408750 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6815c20-c811-4fe7-84bc-ab0385704673 admin admin] RBAC: Authorizing `identity:create_grant(project_id=80e8e30abf5c4629bb06bc5e24d23888, user_id=1888662e4b874d40b48594d9ea38a4d5, role_id=d009411496b8452b88e77ffb8db015a7)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:30:03.411467 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6815c20-c811-4fe7-84bc-ab0385704673 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:30:03.429224 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-f2051ee3-253c-4b12-a5df-aca652a4cbb1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d129b0cc9604fc48a269b4465585ac3'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:03.442025 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-f2051ee3-253c-4b12-a5df-aca652a4cbb1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d129b0cc9604fc48a269b4465585ac3'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:03.442715 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-f2051ee3-253c-4b12-a5df-aca652a4cbb1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d129b0cc9604fc48a269b4465585ac3'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:03.451321 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-f2051ee3-253c-4b12-a5df-aca652a4cbb1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d129b0cc9604fc48a269b4465585ac3'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:03.496704 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1308/2621] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 01:30:03 2022] PUT /identity/v3/projects/80e8e30abf5c4629bb06bc5e24d23888/users/1888662e4b874d40b48594d9ea38a4d5/roles/d009411496b8452b88e77ffb8db015a7 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 01:30:03.500434 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6815c20-c811-4fe7-84bc-ab0385704673 admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:30:03.502043 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-a6815c20-c811-4fe7-84bc-ab0385704673 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:30:03.513110 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-f2051ee3-253c-4b12-a5df-aca652a4cbb1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d129b0cc9604fc48a269b4465585ac3'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:03.519799 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1314/2622] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 01:30:03 2022] DELETE /identity/v3/users/0d129b0cc9604fc48a269b4465585ac3 => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 01:30:03.524213 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35eea59-f92a-483b-af8a-afa46b6d984d None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:03.524582 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35eea59-f92a-483b-af8a-afa46b6d984d None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:03.524893 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35eea59-f92a-483b-af8a-afa46b6d984d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:03.543785 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-c35eea59-f92a-483b-af8a-afa46b6d984d None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:30:03.545022 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-c35eea59-f92a-483b-af8a-afa46b6d984d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:30:03.571913 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6815c20-c811-4fe7-84bc-ab0385704673 admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:30:03.605626 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1315/2623] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:30:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 01:30:03.615515 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35eea59-f92a-483b-af8a-afa46b6d984d None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:30:03.617127 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-c35eea59-f92a-483b-af8a-afa46b6d984d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:30:03.629923 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227a0bb1-8ac5-4ae2-acc0-5e9cbd24c873 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:30:03.630562 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227a0bb1-8ac5-4ae2-acc0-5e9cbd24c873 admin admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:03.630844 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227a0bb1-8ac5-4ae2-acc0-5e9cbd24c873 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:03.631274 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227a0bb1-8ac5-4ae2-acc0-5e9cbd24c873 admin admin] PATH_INFO: `/v3/roles` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:03.632035 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227a0bb1-8ac5-4ae2-acc0-5e9cbd24c873 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:30:03.633768 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227a0bb1-8ac5-4ae2-acc0-5e9cbd24c873 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:30:03.643854 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1309/2624] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:30:03 2022] GET /identity/v3/roles => generated 1420 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 01:30:03.648230 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227a0bb1-8ac5-4ae2-acc0-5e9cbd24c873 admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:30:03.658602 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227a0bb1-8ac5-4ae2-acc0-5e9cbd24c873 admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:30:03.663434 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9155cc1e-358b-4d5f-9992-d54027284ac9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:30:03.664117 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9155cc1e-358b-4d5f-9992-d54027284ac9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:03.664441 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9155cc1e-358b-4d5f-9992-d54027284ac9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:03.664737 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9155cc1e-358b-4d5f-9992-d54027284ac9 admin admin] PATH_INFO: `/v3/projects/80e8e30abf5c4629bb06bc5e24d23888/users/1888662e4b874d40b48594d9ea38a4d5/roles/d009411496b8452b88e77ffb8db015a7` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:03.667164 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9155cc1e-358b-4d5f-9992-d54027284ac9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=80e8e30abf5c4629bb06bc5e24d23888, user_id=1888662e4b874d40b48594d9ea38a4d5, role_id=d009411496b8452b88e77ffb8db015a7)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:30:03.669839 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9155cc1e-358b-4d5f-9992-d54027284ac9 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:30:03.674650 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35eea59-f92a-483b-af8a-afa46b6d984d None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:30:03.679899 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1310/2625] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 01:30:03 2022] PUT /identity/v3/projects/80e8e30abf5c4629bb06bc5e24d23888/users/1888662e4b874d40b48594d9ea38a4d5/roles/d009411496b8452b88e77ffb8db015a7 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:30:03.684369 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:30:03.691901 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1e894f-d09b-4d7e-b91b-9f6c90adf88a None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:03.692335 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1e894f-d09b-4d7e-b91b-9f6c90adf88a None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:03.692617 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1e894f-d09b-4d7e-b91b-9f6c90adf88a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:03.720818 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-7e1e894f-d09b-4d7e-b91b-9f6c90adf88a None None] MFA Rules not processed for user `1888662e4b874d40b48594d9ea38a4d5`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:30:03.722211 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-7e1e894f-d09b-4d7e-b91b-9f6c90adf88a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:30:03.729666 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3eed08-882b-4857-b3a8-c1d2a16953b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0817c3320d5d41d8bba0550c8ae8dd60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:30:03.730290 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3eed08-882b-4857-b3a8-c1d2a16953b3 service neutron] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:03.730622 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3eed08-882b-4857-b3a8-c1d2a16953b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:03.730896 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3eed08-882b-4857-b3a8-c1d2a16953b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:03.756592 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:30:03.802802 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3eed08-882b-4857-b3a8-c1d2a16953b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:30:03.804763 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3eed08-882b-4857-b3a8-c1d2a16953b3 service neutron] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:30:03.811709 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1311/2626] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:30:03 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:30:03.816185 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1316/2627] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:30:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 01:30:03.824569 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1e894f-d09b-4d7e-b91b-9f6c90adf88a None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:30:03.826555 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-7e1e894f-d09b-4d7e-b91b-9f6c90adf88a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:30:03.885018 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1e894f-d09b-4d7e-b91b-9f6c90adf88a None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:30:03.888767 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbabbf8f-64ee-4fb1-b1e4-2609b46540d4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5c8fc69c518412ba1aad61a9f4f558c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:30:03.889533 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbabbf8f-64ee-4fb1-b1e4-2609b46540d4 service cinder] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:03.889810 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbabbf8f-64ee-4fb1-b1e4-2609b46540d4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:03.890080 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbabbf8f-64ee-4fb1-b1e4-2609b46540d4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:03.906379 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbabbf8f-64ee-4fb1-b1e4-2609b46540d4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:30:03.907943 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbabbf8f-64ee-4fb1-b1e4-2609b46540d4 service cinder] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:30:03.919864 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1312/2628] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:30:03 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:30:04.216982 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3eed08-882b-4857-b3a8-c1d2a16953b3 service neutron] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:30:04.216982 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-6b3eed08-882b-4857-b3a8-c1d2a16953b3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:30:04.238330 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3eed08-882b-4857-b3a8-c1d2a16953b3 service neutron] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:30:04.244303 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba2cefc-fa49-4a95-9173-2a3158698282 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:30:04.244974 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba2cefc-fa49-4a95-9173-2a3158698282 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:04.245259 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba2cefc-fa49-4a95-9173-2a3158698282 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:04.245587 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba2cefc-fa49-4a95-9173-2a3158698282 admin admin] PATH_INFO: `/v3/projects/940c6050a62a42429fbf173565acffa5` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:04.246721 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba2cefc-fa49-4a95-9173-2a3158698282 admin admin] RBAC: Authorizing `identity:delete_project(project_id=940c6050a62a42429fbf173565acffa5)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:30:04.248670 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba2cefc-fa49-4a95-9173-2a3158698282 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:30:04.277523 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbabbf8f-64ee-4fb1-b1e4-2609b46540d4 service cinder] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:30:04.286084 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbabbf8f-64ee-4fb1-b1e4-2609b46540d4 service cinder] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:30:04.291650 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9728e0bb-8ff0-49cc-8f64-7422d0b9a274 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5c8fc69c518412ba1aad61a9f4f558c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:30:04.291650 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9728e0bb-8ff0-49cc-8f64-7422d0b9a274 service cinder] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:04.291650 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9728e0bb-8ff0-49cc-8f64-7422d0b9a274 service cinder] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:04.291945 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9728e0bb-8ff0-49cc-8f64-7422d0b9a274 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:04.293550 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-9728e0bb-8ff0-49cc-8f64-7422d0b9a274 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:30:04.348189 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-fba2cefc-fa49-4a95-9173-2a3158698282 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '940c6050a62a42429fbf173565acffa5', 'request_id': 'req-fba2cefc-fa49-4a95-9173-2a3158698282'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:04.353562 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9728e0bb-8ff0-49cc-8f64-7422d0b9a274 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:30:04.353758 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-fba2cefc-fa49-4a95-9173-2a3158698282 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '940c6050a62a42429fbf173565acffa5', 'request_id': 'req-fba2cefc-fa49-4a95-9173-2a3158698282'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:04.355851 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9728e0bb-8ff0-49cc-8f64-7422d0b9a274 service cinder] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:30:04.365019 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1313/2629] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:30:04 2022] GET /identity/v3/auth/tokens => generated 3284 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 01:30:04.421836 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-fba2cefc-fa49-4a95-9173-2a3158698282 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '940c6050a62a42429fbf173565acffa5', 'request_id': 'req-fba2cefc-fa49-4a95-9173-2a3158698282'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:04.434936 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-fba2cefc-fa49-4a95-9173-2a3158698282 admin admin] The token cache is being invalidate because project 940c6050a62a42429fbf173565acffa5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81750) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 01:30:04.434936 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-fba2cefc-fa49-4a95-9173-2a3158698282 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:04.434936 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-fba2cefc-fa49-4a95-9173-2a3158698282 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:04.436163 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1317/2630] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 01:30:04 2022] DELETE /identity/v3/projects/940c6050a62a42429fbf173565acffa5 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:30:04.441217 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0299641-0cca-42f4-bc69-bb451769b564 None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:04.441511 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0299641-0cca-42f4-bc69-bb451769b564 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:04.441796 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0299641-0cca-42f4-bc69-bb451769b564 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:04.463122 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-a0299641-0cca-42f4-bc69-bb451769b564 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:30:04.464979 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-a0299641-0cca-42f4-bc69-bb451769b564 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:30:04.527249 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1314/2631] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:30:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:30:04.535694 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3b2b68-7ca3-4647-857f-90f369ab8670 None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:04.535989 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3b2b68-7ca3-4647-857f-90f369ab8670 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:04.536271 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3b2b68-7ca3-4647-857f-90f369ab8670 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:04.556069 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-4f3b2b68-7ca3-4647-857f-90f369ab8670 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:30:04.557321 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-4f3b2b68-7ca3-4647-857f-90f369ab8670 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:30:04.565681 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1318/2632] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:30:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:30:04.570113 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0299641-0cca-42f4-bc69-bb451769b564 None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:30:04.575507 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:30:04.581119 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0299641-0cca-42f4-bc69-bb451769b564 None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:30:04.585780 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24347bd4-256b-4802-9f8b-915113dc9cf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:30:04.586366 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24347bd4-256b-4802-9f8b-915113dc9cf0 admin admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:04.586645 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24347bd4-256b-4802-9f8b-915113dc9cf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:04.586919 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24347bd4-256b-4802-9f8b-915113dc9cf0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:04.587553 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24347bd4-256b-4802-9f8b-915113dc9cf0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:30:04.587865 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24347bd4-256b-4802-9f8b-915113dc9cf0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:30:04.590060 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24347bd4-256b-4802-9f8b-915113dc9cf0 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:30:04.601267 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1315/2633] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Aug 31 01:30:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:30:04.605596 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3b2b68-7ca3-4647-857f-90f369ab8670 None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:30:04.615014 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3b2b68-7ca3-4647-857f-90f369ab8670 None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:30:04.619734 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a111c73-1094-4b80-9de4-50f925d9683a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:30:04.620328 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a111c73-1094-4b80-9de4-50f925d9683a admin admin] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:04.620650 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a111c73-1094-4b80-9de4-50f925d9683a admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:04.620972 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a111c73-1094-4b80-9de4-50f925d9683a admin admin] PATH_INFO: `/v3/projects` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:04.621628 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a111c73-1094-4b80-9de4-50f925d9683a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:30:04.623524 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a111c73-1094-4b80-9de4-50f925d9683a admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:30:04.708685 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1319/2634] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 01:30:04 2022] POST /identity/v3/projects => generated 385 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 01:30:04.713371 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24347bd4-256b-4802-9f8b-915113dc9cf0 admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:30:04.723364 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24347bd4-256b-4802-9f8b-915113dc9cf0 admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:30:04.780362 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000d0b25-d9f2-4367-b149-a380c597d87a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:30:04.780932 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000d0b25-d9f2-4367-b149-a380c597d87a admin admin] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:04.781212 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000d0b25-d9f2-4367-b149-a380c597d87a admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:04.781487 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000d0b25-d9f2-4367-b149-a380c597d87a admin admin] PATH_INFO: `/v3/users` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:04.782155 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000d0b25-d9f2-4367-b149-a380c597d87a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:30:04.784062 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000d0b25-d9f2-4367-b149-a380c597d87a admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:30:04.873227 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1316/2635] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 01:30:04 2022] POST /identity/v3/users => generated 293 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 01:30:04.877225 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a111c73-1094-4b80-9de4-50f925d9683a admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:30:04.887354 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a111c73-1094-4b80-9de4-50f925d9683a admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:30:04.892869 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46e4dd4-904e-41a8-ab11-43da62595b88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:30:04.893479 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46e4dd4-904e-41a8-ab11-43da62595b88 admin admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:04.893758 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46e4dd4-904e-41a8-ab11-43da62595b88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:04.894065 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46e4dd4-904e-41a8-ab11-43da62595b88 admin admin] PATH_INFO: `/v3/roles` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:04.894695 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46e4dd4-904e-41a8-ab11-43da62595b88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:30:04.896399 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46e4dd4-904e-41a8-ab11-43da62595b88 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:30:04.910887 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1320/2636] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:30:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:30:04.912097 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000d0b25-d9f2-4367-b149-a380c597d87a admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:30:04.922972 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000d0b25-d9f2-4367-b149-a380c597d87a admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:30:04.927786 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb036347-607f-440c-9ad8-3bb3b6d440b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:30:04.928392 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb036347-607f-440c-9ad8-3bb3b6d440b9 admin admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:04.928782 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb036347-607f-440c-9ad8-3bb3b6d440b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:04.929043 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb036347-607f-440c-9ad8-3bb3b6d440b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:04.929689 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb036347-607f-440c-9ad8-3bb3b6d440b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:30:04.931294 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb036347-607f-440c-9ad8-3bb3b6d440b9 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:30:04.942611 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1317/2637] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:30:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:30:04.947717 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46e4dd4-904e-41a8-ab11-43da62595b88 admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:30:04.969173 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46e4dd4-904e-41a8-ab11-43da62595b88 admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:30:04.972862 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5598e53-0a9f-4a3a-878f-355e00f5d697 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:30:04.973542 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5598e53-0a9f-4a3a-878f-355e00f5d697 admin admin] REQUEST_METHOD: `PUT` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:04.973825 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5598e53-0a9f-4a3a-878f-355e00f5d697 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:04.974103 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5598e53-0a9f-4a3a-878f-355e00f5d697 admin admin] PATH_INFO: `/v3/projects/9d0bfc7ca4994ce9a9b004238fdcec47/users/b0a373802c2a47dbaea0545ac7de84fc/roles/d0b9768f4d014766b9326387f2e11f34` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:04.976674 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:30:04.999162 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5598e53-0a9f-4a3a-878f-355e00f5d697 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d0bfc7ca4994ce9a9b004238fdcec47, user_id=b0a373802c2a47dbaea0545ac7de84fc, role_id=d0b9768f4d014766b9326387f2e11f34)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:30:05.002016 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5598e53-0a9f-4a3a-878f-355e00f5d697 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:30:05.100956 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1321/2638] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 01:30:04 2022] PUT /identity/v3/projects/9d0bfc7ca4994ce9a9b004238fdcec47/users/b0a373802c2a47dbaea0545ac7de84fc/roles/d0b9768f4d014766b9326387f2e11f34 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:30:05.111201 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2f2f66-2aa1-4228-be01-61a850a88279 None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:05.111496 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2f2f66-2aa1-4228-be01-61a850a88279 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:05.111781 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2f2f66-2aa1-4228-be01-61a850a88279 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:05.140937 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-7d2f2f66-2aa1-4228-be01-61a850a88279 None None] MFA Rules not processed for user `b0a373802c2a47dbaea0545ac7de84fc`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:30:05.142173 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-7d2f2f66-2aa1-4228-be01-61a850a88279 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:30:05.210164 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1318/2639] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:30:05 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:30:05.222052 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5598e53-0a9f-4a3a-878f-355e00f5d697 admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:30:05.223782 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-e5598e53-0a9f-4a3a-878f-355e00f5d697 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:30:05.281055 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5598e53-0a9f-4a3a-878f-355e00f5d697 admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:30:05.285047 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c9394e-b271-4b1e-8212-c4def7d3daed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5c8fc69c518412ba1aad61a9f4f558c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:30:05.285654 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c9394e-b271-4b1e-8212-c4def7d3daed service cinder] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:05.285936 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c9394e-b271-4b1e-8212-c4def7d3daed service cinder] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:05.286213 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c9394e-b271-4b1e-8212-c4def7d3daed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:05.295835 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c9394e-b271-4b1e-8212-c4def7d3daed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:30:05.297331 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c9394e-b271-4b1e-8212-c4def7d3daed service cinder] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:30:05.309511 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1322/2640] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:30:05 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 01:30:15.207266 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=71,keystone:INSERT=3 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:30:15.758676 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=33,keystone:INSERT=1 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:30:20.601121 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2f2f66-2aa1-4228-be01-61a850a88279 None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:30:20.602695 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-7d2f2f66-2aa1-4228-be01-61a850a88279 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:30:20.693707 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2f2f66-2aa1-4228-be01-61a850a88279 None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:30:20.698956 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c907a14e-e8f0-4c13-843e-f62599a13d77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:30:20.699685 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c907a14e-e8f0-4c13-843e-f62599a13d77 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:20.700000 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c907a14e-e8f0-4c13-843e-f62599a13d77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:20.700283 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c907a14e-e8f0-4c13-843e-f62599a13d77 admin admin] PATH_INFO: `/v3/users/e1fdb3fa8ebd4e9eae78e1abb91842e7` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:20.701438 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c907a14e-e8f0-4c13-843e-f62599a13d77 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e1fdb3fa8ebd4e9eae78e1abb91842e7)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:30:20.703416 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c907a14e-e8f0-4c13-843e-f62599a13d77 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:30:20.845873 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-c907a14e-e8f0-4c13-843e-f62599a13d77 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e1fdb3fa8ebd4e9eae78e1abb91842e7'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:20.912551 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-c907a14e-e8f0-4c13-843e-f62599a13d77 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e1fdb3fa8ebd4e9eae78e1abb91842e7'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:20.913226 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-c907a14e-e8f0-4c13-843e-f62599a13d77 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e1fdb3fa8ebd4e9eae78e1abb91842e7'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:20.921337 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-c907a14e-e8f0-4c13-843e-f62599a13d77 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e1fdb3fa8ebd4e9eae78e1abb91842e7'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:20.922023 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-c907a14e-e8f0-4c13-843e-f62599a13d77 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e1fdb3fa8ebd4e9eae78e1abb91842e7'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:20.941579 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1319/2641] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 01:30:20 2022] DELETE /identity/v3/users/e1fdb3fa8ebd4e9eae78e1abb91842e7 => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:30:20.949596 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dc8825-3859-4c57-a8f5-dcd708d683e7 None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:20.949596 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dc8825-3859-4c57-a8f5-dcd708d683e7 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:20.949596 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dc8825-3859-4c57-a8f5-dcd708d683e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:20.967451 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-69dc8825-3859-4c57-a8f5-dcd708d683e7 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:30:20.968760 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-69dc8825-3859-4c57-a8f5-dcd708d683e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:30:21.028496 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1323/2642] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:30:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:30:21.038374 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c907a14e-e8f0-4c13-843e-f62599a13d77 admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:30:21.041322 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-c907a14e-e8f0-4c13-843e-f62599a13d77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:30:21.101136 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c907a14e-e8f0-4c13-843e-f62599a13d77 admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:30:21.105431 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef81c686-8506-4cc6-92a0-f7f7750b3733 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0817c3320d5d41d8bba0550c8ae8dd60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:30:21.106051 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef81c686-8506-4cc6-92a0-f7f7750b3733 service neutron] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:21.110251 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef81c686-8506-4cc6-92a0-f7f7750b3733 service neutron] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:21.110251 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef81c686-8506-4cc6-92a0-f7f7750b3733 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:21.115593 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef81c686-8506-4cc6-92a0-f7f7750b3733 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:30:21.117117 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef81c686-8506-4cc6-92a0-f7f7750b3733 service neutron] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:30:21.127140 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1320/2643] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:30:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:30:21.486164 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69dc8825-3859-4c57-a8f5-dcd708d683e7 None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:30:21.487582 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-69dc8825-3859-4c57-a8f5-dcd708d683e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:30:21.499817 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69dc8825-3859-4c57-a8f5-dcd708d683e7 None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:30:21.504465 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b38342a-6dea-4bc4-9897-0bca356e7112 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:30:21.505129 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b38342a-6dea-4bc4-9897-0bca356e7112 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:21.505413 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b38342a-6dea-4bc4-9897-0bca356e7112 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:21.505690 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b38342a-6dea-4bc4-9897-0bca356e7112 admin admin] PATH_INFO: `/v3/projects/a3f745e0cc9c4f9b91aedb77410ca019` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:21.506878 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b38342a-6dea-4bc4-9897-0bca356e7112 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a3f745e0cc9c4f9b91aedb77410ca019)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:30:21.508933 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b38342a-6dea-4bc4-9897-0bca356e7112 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:30:21.605158 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-8b38342a-6dea-4bc4-9897-0bca356e7112 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a3f745e0cc9c4f9b91aedb77410ca019', 'request_id': 'req-8b38342a-6dea-4bc4-9897-0bca356e7112'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:21.610688 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-8b38342a-6dea-4bc4-9897-0bca356e7112 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a3f745e0cc9c4f9b91aedb77410ca019', 'request_id': 'req-8b38342a-6dea-4bc4-9897-0bca356e7112'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:21.666682 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-8b38342a-6dea-4bc4-9897-0bca356e7112 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a3f745e0cc9c4f9b91aedb77410ca019', 'request_id': 'req-8b38342a-6dea-4bc4-9897-0bca356e7112'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:21.677855 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-8b38342a-6dea-4bc4-9897-0bca356e7112 admin admin] The token cache is being invalidate because project a3f745e0cc9c4f9b91aedb77410ca019 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81750) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 01:30:21.680602 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-8b38342a-6dea-4bc4-9897-0bca356e7112 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:21.680602 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-8b38342a-6dea-4bc4-9897-0bca356e7112 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:21.680602 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1324/2644] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 01:30:21 2022] DELETE /identity/v3/projects/a3f745e0cc9c4f9b91aedb77410ca019 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:30:21.684943 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef81c686-8506-4cc6-92a0-f7f7750b3733 service neutron] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:30:21.695293 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef81c686-8506-4cc6-92a0-f7f7750b3733 service neutron] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:30:21.703393 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b38342a-6dea-4bc4-9897-0bca356e7112 admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:30:21.704980 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-8b38342a-6dea-4bc4-9897-0bca356e7112 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:30:21.718236 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:30:21.755653 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61aa970a-c97a-44d3-8c6c-62a6c3b0e0b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:30:21.756210 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61aa970a-c97a-44d3-8c6c-62a6c3b0e0b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:21.756499 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61aa970a-c97a-44d3-8c6c-62a6c3b0e0b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:21.756813 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61aa970a-c97a-44d3-8c6c-62a6c3b0e0b4 admin admin] PATH_INFO: `/v3/users/1888662e4b874d40b48594d9ea38a4d5` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:21.757934 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61aa970a-c97a-44d3-8c6c-62a6c3b0e0b4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1888662e4b874d40b48594d9ea38a4d5)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:30:21.759860 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61aa970a-c97a-44d3-8c6c-62a6c3b0e0b4 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:30:21.770401 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b38342a-6dea-4bc4-9897-0bca356e7112 admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:30:21.775354 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c657eb-986b-4022-ba9f-b3fb97818a8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:30:21.776839 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c657eb-986b-4022-ba9f-b3fb97818a8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:21.777223 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c657eb-986b-4022-ba9f-b3fb97818a8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:21.777598 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c657eb-986b-4022-ba9f-b3fb97818a8c admin admin] PATH_INFO: `/v3/users/b0a373802c2a47dbaea0545ac7de84fc` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:21.778860 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c657eb-986b-4022-ba9f-b3fb97818a8c admin admin] RBAC: Authorizing `identity:delete_user(user_id=b0a373802c2a47dbaea0545ac7de84fc)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:30:21.780797 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c657eb-986b-4022-ba9f-b3fb97818a8c admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:30:21.867778 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:30:21.937890 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-61aa970a-c97a-44d3-8c6c-62a6c3b0e0b4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1888662e4b874d40b48594d9ea38a4d5'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:21.961538 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-34c657eb-986b-4022-ba9f-b3fb97818a8c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b0a373802c2a47dbaea0545ac7de84fc'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:21.973130 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-34c657eb-986b-4022-ba9f-b3fb97818a8c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b0a373802c2a47dbaea0545ac7de84fc'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:21.973824 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-34c657eb-986b-4022-ba9f-b3fb97818a8c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b0a373802c2a47dbaea0545ac7de84fc'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:21.980500 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-34c657eb-986b-4022-ba9f-b3fb97818a8c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b0a373802c2a47dbaea0545ac7de84fc'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:21.987038 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-61aa970a-c97a-44d3-8c6c-62a6c3b0e0b4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1888662e4b874d40b48594d9ea38a4d5'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:21.987658 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-61aa970a-c97a-44d3-8c6c-62a6c3b0e0b4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1888662e4b874d40b48594d9ea38a4d5'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:21.995920 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-61aa970a-c97a-44d3-8c6c-62a6c3b0e0b4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1888662e4b874d40b48594d9ea38a4d5'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:21.996586 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-61aa970a-c97a-44d3-8c6c-62a6c3b0e0b4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1888662e4b874d40b48594d9ea38a4d5'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:22.017126 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1321/2645] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 01:30:21 2022] DELETE /identity/v3/users/1888662e4b874d40b48594d9ea38a4d5 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:30:22.044060 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-34c657eb-986b-4022-ba9f-b3fb97818a8c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b0a373802c2a47dbaea0545ac7de84fc'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:22.050601 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1325/2646] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 01:30:21 2022] DELETE /identity/v3/users/b0a373802c2a47dbaea0545ac7de84fc => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 01:30:22.055071 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f054c6d1-cb0c-4b23-8060-2eef0502f1d4 None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:22.055354 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f054c6d1-cb0c-4b23-8060-2eef0502f1d4 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:22.055636 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f054c6d1-cb0c-4b23-8060-2eef0502f1d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:22.075976 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-f054c6d1-cb0c-4b23-8060-2eef0502f1d4 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:30:22.077321 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-f054c6d1-cb0c-4b23-8060-2eef0502f1d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:30:22.139150 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1322/2647] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:30:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:30:22.156244 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c657eb-986b-4022-ba9f-b3fb97818a8c admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:30:22.157747 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-34c657eb-986b-4022-ba9f-b3fb97818a8c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:30:22.226194 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c657eb-986b-4022-ba9f-b3fb97818a8c admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:30:22.228892 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc31bd0-95bd-4c17-93cd-7f411ed8e44a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0817c3320d5d41d8bba0550c8ae8dd60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:30:22.229607 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc31bd0-95bd-4c17-93cd-7f411ed8e44a service neutron] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:22.229964 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc31bd0-95bd-4c17-93cd-7f411ed8e44a service neutron] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:22.230464 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc31bd0-95bd-4c17-93cd-7f411ed8e44a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:22.240323 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc31bd0-95bd-4c17-93cd-7f411ed8e44a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:30:22.241863 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc31bd0-95bd-4c17-93cd-7f411ed8e44a service neutron] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:30:22.252776 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1326/2648] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:30:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:30:22.389012 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f054c6d1-cb0c-4b23-8060-2eef0502f1d4 None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:30:22.390458 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-f054c6d1-cb0c-4b23-8060-2eef0502f1d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:30:22.402446 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f054c6d1-cb0c-4b23-8060-2eef0502f1d4 None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:30:22.407561 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d3c30a-28b8-4f34-9f3d-c3fa5270df74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:30:22.408207 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d3c30a-28b8-4f34-9f3d-c3fa5270df74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:22.408494 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d3c30a-28b8-4f34-9f3d-c3fa5270df74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:22.408808 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d3c30a-28b8-4f34-9f3d-c3fa5270df74 admin admin] PATH_INFO: `/v3/projects/80e8e30abf5c4629bb06bc5e24d23888` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:22.409956 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d3c30a-28b8-4f34-9f3d-c3fa5270df74 admin admin] RBAC: Authorizing `identity:delete_project(project_id=80e8e30abf5c4629bb06bc5e24d23888)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:30:22.411894 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d3c30a-28b8-4f34-9f3d-c3fa5270df74 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:30:22.528137 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:30:22.537003 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-19d3c30a-28b8-4f34-9f3d-c3fa5270df74 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '80e8e30abf5c4629bb06bc5e24d23888', 'request_id': 'req-19d3c30a-28b8-4f34-9f3d-c3fa5270df74'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:22.543642 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-19d3c30a-28b8-4f34-9f3d-c3fa5270df74 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '80e8e30abf5c4629bb06bc5e24d23888', 'request_id': 'req-19d3c30a-28b8-4f34-9f3d-c3fa5270df74'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:22.550173 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-19d3c30a-28b8-4f34-9f3d-c3fa5270df74 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '80e8e30abf5c4629bb06bc5e24d23888', 'request_id': 'req-19d3c30a-28b8-4f34-9f3d-c3fa5270df74'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:22.621208 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-19d3c30a-28b8-4f34-9f3d-c3fa5270df74 admin admin] The token cache is being invalidate because project 80e8e30abf5c4629bb06bc5e24d23888 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81749) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 01:30:22.621501 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-19d3c30a-28b8-4f34-9f3d-c3fa5270df74 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:22.622153 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-19d3c30a-28b8-4f34-9f3d-c3fa5270df74 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:22.623643 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1323/2649] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 01:30:22 2022] DELETE /identity/v3/projects/80e8e30abf5c4629bb06bc5e24d23888 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 01:30:22.628780 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9513955f-aab6-4548-8297-657596d3fa6c None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:22.629075 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9513955f-aab6-4548-8297-657596d3fa6c None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:22.629357 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9513955f-aab6-4548-8297-657596d3fa6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:22.648644 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-9513955f-aab6-4548-8297-657596d3fa6c None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:30:22.649887 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-9513955f-aab6-4548-8297-657596d3fa6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:30:22.687483 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d3c30a-28b8-4f34-9f3d-c3fa5270df74 admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:30:22.687483 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-19d3c30a-28b8-4f34-9f3d-c3fa5270df74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:30:22.731061 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1327/2650] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:30:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:30:22.732986 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23b0b43-5348-471d-bdad-af766d9af285 None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:22.733319 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23b0b43-5348-471d-bdad-af766d9af285 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:22.733666 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23b0b43-5348-471d-bdad-af766d9af285 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:22.755691 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-d23b0b43-5348-471d-bdad-af766d9af285 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:30:22.757109 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-d23b0b43-5348-471d-bdad-af766d9af285 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:30:22.765505 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1328/2651] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:30:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:30:22.769854 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23b0b43-5348-471d-bdad-af766d9af285 None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:30:22.775582 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d3c30a-28b8-4f34-9f3d-c3fa5270df74 admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:30:22.779912 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23b0b43-5348-471d-bdad-af766d9af285 None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:30:22.781033 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb13794-5ad0-493f-b65d-c98b3449691e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:30:22.781705 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb13794-5ad0-493f-b65d-c98b3449691e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:22.781990 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb13794-5ad0-493f-b65d-c98b3449691e admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:22.782271 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb13794-5ad0-493f-b65d-c98b3449691e admin admin] PATH_INFO: `/v3/projects/9d0bfc7ca4994ce9a9b004238fdcec47` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:22.783398 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb13794-5ad0-493f-b65d-c98b3449691e admin admin] RBAC: Authorizing `identity:delete_project(project_id=9d0bfc7ca4994ce9a9b004238fdcec47)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:30:22.784755 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9674298e-30ae-4115-81fd-429167919bc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:30:22.785351 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9674298e-30ae-4115-81fd-429167919bc4 admin admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:22.785528 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb13794-5ad0-493f-b65d-c98b3449691e admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:30:22.785626 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9674298e-30ae-4115-81fd-429167919bc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:22.785896 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9674298e-30ae-4115-81fd-429167919bc4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:22.786537 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9674298e-30ae-4115-81fd-429167919bc4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:30:22.786851 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9674298e-30ae-4115-81fd-429167919bc4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:30:22.788425 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9674298e-30ae-4115-81fd-429167919bc4 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:30:22.804168 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1329/2652] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Aug 31 01:30:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 01:30:22.805209 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9674298e-30ae-4115-81fd-429167919bc4 admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:30:22.815352 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9674298e-30ae-4115-81fd-429167919bc4 admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:30:22.820226 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de8673a-d4d6-4e98-b5eb-7b2dd5d61bc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:30:22.825929 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de8673a-d4d6-4e98-b5eb-7b2dd5d61bc9 admin admin] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:22.825929 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de8673a-d4d6-4e98-b5eb-7b2dd5d61bc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:22.825929 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de8673a-d4d6-4e98-b5eb-7b2dd5d61bc9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:22.825929 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de8673a-d4d6-4e98-b5eb-7b2dd5d61bc9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:30:22.825929 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de8673a-d4d6-4e98-b5eb-7b2dd5d61bc9 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:30:22.913723 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-4bb13794-5ad0-493f-b65d-c98b3449691e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9d0bfc7ca4994ce9a9b004238fdcec47', 'request_id': 'req-4bb13794-5ad0-493f-b65d-c98b3449691e'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:22.920432 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-4bb13794-5ad0-493f-b65d-c98b3449691e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9d0bfc7ca4994ce9a9b004238fdcec47', 'request_id': 'req-4bb13794-5ad0-493f-b65d-c98b3449691e'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:22.927484 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-4bb13794-5ad0-493f-b65d-c98b3449691e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9d0bfc7ca4994ce9a9b004238fdcec47', 'request_id': 'req-4bb13794-5ad0-493f-b65d-c98b3449691e'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:22.931772 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1330/2653] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 01:30:22 2022] POST /identity/v3/projects => generated 371 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 01:30:22.932874 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de8673a-d4d6-4e98-b5eb-7b2dd5d61bc9 admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:30:22.939014 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:DELETE=6,keystone:SELECT=92,keystone:INSERT=2 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:30:22.987343 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-4bb13794-5ad0-493f-b65d-c98b3449691e admin admin] The token cache is being invalidate because project 9d0bfc7ca4994ce9a9b004238fdcec47 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81749) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 01:30:22.987731 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-4bb13794-5ad0-493f-b65d-c98b3449691e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:22.988495 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-4bb13794-5ad0-493f-b65d-c98b3449691e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:22.990086 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1324/2654] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 01:30:22 2022] DELETE /identity/v3/projects/9d0bfc7ca4994ce9a9b004238fdcec47 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 01:30:22.994965 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659afa57-4d99-4f8a-ac67-711573656c45 None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:22.995258 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659afa57-4d99-4f8a-ac67-711573656c45 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:22.995542 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659afa57-4d99-4f8a-ac67-711573656c45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:22.999543 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de8673a-d4d6-4e98-b5eb-7b2dd5d61bc9 admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:30:23.004605 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c2a1e6-542d-4a3a-bc64-b0192dc4b450 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:30:23.005264 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c2a1e6-542d-4a3a-bc64-b0192dc4b450 admin admin] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:23.005549 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c2a1e6-542d-4a3a-bc64-b0192dc4b450 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:23.005824 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c2a1e6-542d-4a3a-bc64-b0192dc4b450 admin admin] PATH_INFO: `/v3/users` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:23.006482 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c2a1e6-542d-4a3a-bc64-b0192dc4b450 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:30:23.008410 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c2a1e6-542d-4a3a-bc64-b0192dc4b450 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:30:23.017150 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-659afa57-4d99-4f8a-ac67-711573656c45 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:30:23.018704 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-659afa57-4d99-4f8a-ac67-711573656c45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:30:23.028301 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1325/2655] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:30:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 01:30:23.092962 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ef4347-105b-497c-b0e9-9b9505e5d333 None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:23.093252 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ef4347-105b-497c-b0e9-9b9505e5d333 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:23.093533 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ef4347-105b-497c-b0e9-9b9505e5d333 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:23.114071 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-29ef4347-105b-497c-b0e9-9b9505e5d333 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:30:23.115547 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-29ef4347-105b-497c-b0e9-9b9505e5d333 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:30:23.123874 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1326/2656] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:30:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 01:30:23.128172 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ef4347-105b-497c-b0e9-9b9505e5d333 None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:30:23.138055 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ef4347-105b-497c-b0e9-9b9505e5d333 None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:30:23.143230 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503184a2-0416-4279-9c66-64d267644117 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:30:23.143828 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503184a2-0416-4279-9c66-64d267644117 admin admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:23.144116 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503184a2-0416-4279-9c66-64d267644117 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:23.144443 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503184a2-0416-4279-9c66-64d267644117 admin admin] PATH_INFO: `/v3/domains` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:23.145126 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503184a2-0416-4279-9c66-64d267644117 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:30:23.149499 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503184a2-0416-4279-9c66-64d267644117 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:30:23.149499 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503184a2-0416-4279-9c66-64d267644117 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:30:23.149698 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1331/2657] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 01:30:22 2022] POST /identity/v3/users => generated 286 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:30:23.152914 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c2a1e6-542d-4a3a-bc64-b0192dc4b450 admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:30:23.154518 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-97c2a1e6-542d-4a3a-bc64-b0192dc4b450 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:30:23.160214 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1327/2658] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Aug 31 01:30:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 01:30:23.165732 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503184a2-0416-4279-9c66-64d267644117 admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:30:23.167731 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c2a1e6-542d-4a3a-bc64-b0192dc4b450 admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:30:23.172490 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cd684e-9575-4857-854e-a8a2dce40ab3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:30:23.173121 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cd684e-9575-4857-854e-a8a2dce40ab3 admin admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:23.173400 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cd684e-9575-4857-854e-a8a2dce40ab3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:23.173678 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cd684e-9575-4857-854e-a8a2dce40ab3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:23.174317 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cd684e-9575-4857-854e-a8a2dce40ab3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:30:23.175896 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cd684e-9575-4857-854e-a8a2dce40ab3 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:30:23.177502 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503184a2-0416-4279-9c66-64d267644117 admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:30:23.183154 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b64604-a532-415c-ac4c-e694efcb6bd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:30:23.183796 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b64604-a532-415c-ac4c-e694efcb6bd9 admin admin] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:23.184060 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b64604-a532-415c-ac4c-e694efcb6bd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:23.184345 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b64604-a532-415c-ac4c-e694efcb6bd9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:23.185066 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b64604-a532-415c-ac4c-e694efcb6bd9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:30:23.186695 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1332/2659] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:30:23 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 01:30:23.187052 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b64604-a532-415c-ac4c-e694efcb6bd9 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:30:23.190741 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cd684e-9575-4857-854e-a8a2dce40ab3 admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:30:23.204338 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cd684e-9575-4857-854e-a8a2dce40ab3 admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:30:23.210659 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87541d68-276e-475a-bdbd-2056e2d02b7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:30:23.211299 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87541d68-276e-475a-bdbd-2056e2d02b7b admin admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:23.211581 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87541d68-276e-475a-bdbd-2056e2d02b7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:23.211859 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87541d68-276e-475a-bdbd-2056e2d02b7b admin admin] PATH_INFO: `/v3/roles` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:23.212526 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87541d68-276e-475a-bdbd-2056e2d02b7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:30:23.214830 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87541d68-276e-475a-bdbd-2056e2d02b7b admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:30:23.225743 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1333/2660] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:30:23 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 01:30:23.230133 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87541d68-276e-475a-bdbd-2056e2d02b7b admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:30:23.240188 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87541d68-276e-475a-bdbd-2056e2d02b7b admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:30:23.244983 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a956256-acba-4a22-802a-1d5680a33700 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:30:23.245631 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a956256-acba-4a22-802a-1d5680a33700 admin admin] REQUEST_METHOD: `PUT` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:23.245918 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a956256-acba-4a22-802a-1d5680a33700 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:23.247069 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a956256-acba-4a22-802a-1d5680a33700 admin admin] PATH_INFO: `/v3/projects/9850fb6b650349f280454d8870c90e48/users/c61b3e307a2b4a8e9f323cd836090a1b/roles/d0b9768f4d014766b9326387f2e11f34` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:23.263378 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a956256-acba-4a22-802a-1d5680a33700 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9850fb6b650349f280454d8870c90e48, user_id=c61b3e307a2b4a8e9f323cd836090a1b, role_id=d0b9768f4d014766b9326387f2e11f34)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:30:23.266057 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a956256-acba-4a22-802a-1d5680a33700 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:30:23.274455 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1328/2661] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 01:30:23 2022] POST /identity/v3/projects => generated 385 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:30:23.279212 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b64604-a532-415c-ac4c-e694efcb6bd9 admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:30:23.287665 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:30:23.342079 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b64604-a532-415c-ac4c-e694efcb6bd9 admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:30:23.346806 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d061e56-ae6e-483f-be6a-453c7a17ee00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:30:23.347410 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d061e56-ae6e-483f-be6a-453c7a17ee00 admin admin] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:23.347692 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d061e56-ae6e-483f-be6a-453c7a17ee00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:23.347976 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d061e56-ae6e-483f-be6a-453c7a17ee00 admin admin] PATH_INFO: `/v3/users` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:23.348605 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d061e56-ae6e-483f-be6a-453c7a17ee00 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:30:23.350562 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d061e56-ae6e-483f-be6a-453c7a17ee00 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:30:23.358888 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1334/2662] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 01:30:23 2022] PUT /identity/v3/projects/9850fb6b650349f280454d8870c90e48/users/c61b3e307a2b4a8e9f323cd836090a1b/roles/d0b9768f4d014766b9326387f2e11f34 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 01:30:23.370704 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709da92d-9d2b-4f8e-8e42-50ade7c1a89f None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:23.371012 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709da92d-9d2b-4f8e-8e42-50ade7c1a89f None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:23.371288 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709da92d-9d2b-4f8e-8e42-50ade7c1a89f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:23.476676 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1329/2663] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 01:30:23 2022] POST /identity/v3/users => generated 293 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 01:30:23.478182 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d061e56-ae6e-483f-be6a-453c7a17ee00 admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:30:23.489162 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d061e56-ae6e-483f-be6a-453c7a17ee00 admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:30:23.525046 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-709da92d-9d2b-4f8e-8e42-50ade7c1a89f None None] MFA Rules not processed for user `c61b3e307a2b4a8e9f323cd836090a1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:30:23.527050 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-709da92d-9d2b-4f8e-8e42-50ade7c1a89f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:30:23.569151 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782213a8-c6a8-4cd7-be82-0e7d95900f64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:30:23.569747 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782213a8-c6a8-4cd7-be82-0e7d95900f64 admin admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:23.570029 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782213a8-c6a8-4cd7-be82-0e7d95900f64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:23.570301 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782213a8-c6a8-4cd7-be82-0e7d95900f64 admin admin] PATH_INFO: `/v3/roles` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:23.570954 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782213a8-c6a8-4cd7-be82-0e7d95900f64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:30:23.572558 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782213a8-c6a8-4cd7-be82-0e7d95900f64 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:30:23.583678 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1330/2664] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:30:23 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:30:23.589463 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782213a8-c6a8-4cd7-be82-0e7d95900f64 admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:30:23.601973 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782213a8-c6a8-4cd7-be82-0e7d95900f64 admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:30:23.609921 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcca51e9-a178-4aa2-ab90-a6f96946d30e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:30:23.609921 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcca51e9-a178-4aa2-ab90-a6f96946d30e admin admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:23.609921 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcca51e9-a178-4aa2-ab90-a6f96946d30e admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:23.609921 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcca51e9-a178-4aa2-ab90-a6f96946d30e admin admin] PATH_INFO: `/v3/roles` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:23.610584 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcca51e9-a178-4aa2-ab90-a6f96946d30e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:30:23.611193 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcca51e9-a178-4aa2-ab90-a6f96946d30e admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:30:23.623944 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1331/2665] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:30:23 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 01:30:23.627708 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1335/2666] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:30:23 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 01:30:23.628759 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcca51e9-a178-4aa2-ab90-a6f96946d30e admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:30:23.634718 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709da92d-9d2b-4f8e-8e42-50ade7c1a89f None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:30:23.641447 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcca51e9-a178-4aa2-ab90-a6f96946d30e admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:30:23.648154 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30163d92-53ad-45d6-8d6f-2232b2dbe50f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:30:23.648895 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30163d92-53ad-45d6-8d6f-2232b2dbe50f admin admin] REQUEST_METHOD: `PUT` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:23.649227 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30163d92-53ad-45d6-8d6f-2232b2dbe50f admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:23.649359 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709da92d-9d2b-4f8e-8e42-50ade7c1a89f None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:30:23.649513 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30163d92-53ad-45d6-8d6f-2232b2dbe50f admin admin] PATH_INFO: `/v3/projects/765aab3ab0ed4b09815eae61702a4155/users/69cb6d9aff9d4766903e728218f905e3/roles/d0b9768f4d014766b9326387f2e11f34` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:23.654115 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7495f7-f56c-41c2-8017-cb99709ec5a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:30:23.654718 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7495f7-f56c-41c2-8017-cb99709ec5a1 admin admin] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:23.655001 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7495f7-f56c-41c2-8017-cb99709ec5a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:23.655276 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7495f7-f56c-41c2-8017-cb99709ec5a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:23.655904 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7495f7-f56c-41c2-8017-cb99709ec5a1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:30:23.658005 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7495f7-f56c-41c2-8017-cb99709ec5a1 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:30:23.665750 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30163d92-53ad-45d6-8d6f-2232b2dbe50f admin admin] RBAC: Authorizing `identity:create_grant(project_id=765aab3ab0ed4b09815eae61702a4155, user_id=69cb6d9aff9d4766903e728218f905e3, role_id=d0b9768f4d014766b9326387f2e11f34)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:30:23.668438 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30163d92-53ad-45d6-8d6f-2232b2dbe50f admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:30:23.781782 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1332/2667] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 01:30:23 2022] PUT /identity/v3/projects/765aab3ab0ed4b09815eae61702a4155/users/69cb6d9aff9d4766903e728218f905e3/roles/d0b9768f4d014766b9326387f2e11f34 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:30:23.792229 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff65236a-50a6-4908-9a6f-652dc0baa43e None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:23.792633 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff65236a-50a6-4908-9a6f-652dc0baa43e None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:23.792945 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff65236a-50a6-4908-9a6f-652dc0baa43e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:23.812165 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1336/2668] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 01:30:23 2022] POST /identity/v3/projects => generated 373 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:30:23.821431 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7495f7-f56c-41c2-8017-cb99709ec5a1 admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:30:23.823514 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-ff65236a-50a6-4908-9a6f-652dc0baa43e None None] MFA Rules not processed for user `69cb6d9aff9d4766903e728218f905e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:30:23.824766 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-ff65236a-50a6-4908-9a6f-652dc0baa43e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:30:23.843972 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:30:23.879918 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:30:23.891332 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7495f7-f56c-41c2-8017-cb99709ec5a1 admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:30:23.900190 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d9d638-ad94-44a0-9b5f-25c93216c1d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:30:23.900989 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d9d638-ad94-44a0-9b5f-25c93216c1d0 admin admin] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:23.901392 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d9d638-ad94-44a0-9b5f-25c93216c1d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:23.901773 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d9d638-ad94-44a0-9b5f-25c93216c1d0 admin admin] PATH_INFO: `/v3/users` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:23.905648 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1333/2669] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:30:23 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:30:23.906324 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d9d638-ad94-44a0-9b5f-25c93216c1d0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:30:23.906324 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d9d638-ad94-44a0-9b5f-25c93216c1d0 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:30:23.906795 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff65236a-50a6-4908-9a6f-652dc0baa43e None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:30:23.918284 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff65236a-50a6-4908-9a6f-652dc0baa43e None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:30:23.923137 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f641061b-5a95-4934-87f9-d5497d2f6715 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:30:23.932845 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f641061b-5a95-4934-87f9-d5497d2f6715 admin admin] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:23.932845 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f641061b-5a95-4934-87f9-d5497d2f6715 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:23.932845 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f641061b-5a95-4934-87f9-d5497d2f6715 admin admin] PATH_INFO: `/v3/projects` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:23.932845 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f641061b-5a95-4934-87f9-d5497d2f6715 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:30:23.932845 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f641061b-5a95-4934-87f9-d5497d2f6715 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:30:24.020530 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1334/2670] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 01:30:23 2022] POST /identity/v3/projects => generated 385 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 01:30:24.021342 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1337/2671] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 01:30:23 2022] POST /identity/v3/users => generated 293 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:30:24.022630 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f641061b-5a95-4934-87f9-d5497d2f6715 admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:30:24.022761 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d9d638-ad94-44a0-9b5f-25c93216c1d0 admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:30:24.037522 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d9d638-ad94-44a0-9b5f-25c93216c1d0 admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:30:24.093137 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f641061b-5a95-4934-87f9-d5497d2f6715 admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:30:24.097695 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a62d4e-1502-4988-9bc3-9163854fcde2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:30:24.098294 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a62d4e-1502-4988-9bc3-9163854fcde2 admin admin] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:24.098578 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a62d4e-1502-4988-9bc3-9163854fcde2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:24.098849 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a62d4e-1502-4988-9bc3-9163854fcde2 admin admin] PATH_INFO: `/v3/users` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:24.099189 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e679ac5-e29c-4367-9615-c67c126388ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:30:24.099496 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a62d4e-1502-4988-9bc3-9163854fcde2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:30:24.099905 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e679ac5-e29c-4367-9615-c67c126388ed admin admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:24.100196 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e679ac5-e29c-4367-9615-c67c126388ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:24.100472 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e679ac5-e29c-4367-9615-c67c126388ed admin admin] PATH_INFO: `/v3/roles` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:24.101149 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e679ac5-e29c-4367-9615-c67c126388ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:30:24.101619 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a62d4e-1502-4988-9bc3-9163854fcde2 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:30:24.102706 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e679ac5-e29c-4367-9615-c67c126388ed admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:30:24.112694 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1338/2672] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:30:24 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:30:24.116804 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e679ac5-e29c-4367-9615-c67c126388ed admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:30:24.127458 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e679ac5-e29c-4367-9615-c67c126388ed admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:30:24.132347 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca309f1-a419-4c03-8a81-4e4061cc7ad4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:30:24.133025 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca309f1-a419-4c03-8a81-4e4061cc7ad4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:24.133307 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca309f1-a419-4c03-8a81-4e4061cc7ad4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:24.133589 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca309f1-a419-4c03-8a81-4e4061cc7ad4 admin admin] PATH_INFO: `/v3/projects/fdf681bb5f4d4840867692fc68e59fbf/users/7131b6de338c40d19ba55cc403096e13/roles/d009411496b8452b88e77ffb8db015a7` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:24.148610 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca309f1-a419-4c03-8a81-4e4061cc7ad4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fdf681bb5f4d4840867692fc68e59fbf, user_id=7131b6de338c40d19ba55cc403096e13, role_id=d009411496b8452b88e77ffb8db015a7)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:30:24.151327 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca309f1-a419-4c03-8a81-4e4061cc7ad4 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:30:24.176882 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1335/2673] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 01:30:24 2022] POST /identity/v3/users => generated 293 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:30:24.181026 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a62d4e-1502-4988-9bc3-9163854fcde2 admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:30:24.191733 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a62d4e-1502-4988-9bc3-9163854fcde2 admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:30:24.197543 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d231ea2-cadb-4fd5-9985-7412950421f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:30:24.197543 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d231ea2-cadb-4fd5-9985-7412950421f4 admin admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:24.197543 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d231ea2-cadb-4fd5-9985-7412950421f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:24.198380 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d231ea2-cadb-4fd5-9985-7412950421f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:24.198910 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d231ea2-cadb-4fd5-9985-7412950421f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:30:24.200589 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d231ea2-cadb-4fd5-9985-7412950421f4 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:30:24.222749 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1336/2674] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:30:24 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 01:30:24.226720 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d231ea2-cadb-4fd5-9985-7412950421f4 admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:30:24.245914 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d231ea2-cadb-4fd5-9985-7412950421f4 admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:30:24.247963 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1339/2675] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 01:30:24 2022] PUT /identity/v3/projects/fdf681bb5f4d4840867692fc68e59fbf/users/7131b6de338c40d19ba55cc403096e13/roles/d009411496b8452b88e77ffb8db015a7 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:30:24.256414 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca309f1-a419-4c03-8a81-4e4061cc7ad4 admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:30:24.320354 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af02ab1-9964-44c7-8be7-497d6b05ec32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:30:24.321341 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af02ab1-9964-44c7-8be7-497d6b05ec32 admin admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:24.321784 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af02ab1-9964-44c7-8be7-497d6b05ec32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:24.322225 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af02ab1-9964-44c7-8be7-497d6b05ec32 admin admin] PATH_INFO: `/v3/roles` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:24.325057 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af02ab1-9964-44c7-8be7-497d6b05ec32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:30:24.327717 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af02ab1-9964-44c7-8be7-497d6b05ec32 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:30:24.330405 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca309f1-a419-4c03-8a81-4e4061cc7ad4 admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:30:24.335321 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7eef43-2a6c-4e0a-98af-e16563e901a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:30:24.335930 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7eef43-2a6c-4e0a-98af-e16563e901a7 admin admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:24.336215 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7eef43-2a6c-4e0a-98af-e16563e901a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:24.336490 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7eef43-2a6c-4e0a-98af-e16563e901a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:24.337173 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7eef43-2a6c-4e0a-98af-e16563e901a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:30:24.341946 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7eef43-2a6c-4e0a-98af-e16563e901a7 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:30:24.342124 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1337/2676] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:30:24 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 01:30:24.342974 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af02ab1-9964-44c7-8be7-497d6b05ec32 admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:30:24.349737 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1340/2677] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:30:24 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 01:30:24.353569 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7eef43-2a6c-4e0a-98af-e16563e901a7 admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:30:24.356648 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af02ab1-9964-44c7-8be7-497d6b05ec32 admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:30:24.361489 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4c5f26-291a-47ea-9e20-be6bf23310ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:30:24.362263 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4c5f26-291a-47ea-9e20-be6bf23310ad admin admin] REQUEST_METHOD: `PUT` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:24.362614 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4c5f26-291a-47ea-9e20-be6bf23310ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:24.362996 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4c5f26-291a-47ea-9e20-be6bf23310ad admin admin] PATH_INFO: `/v3/projects/38d4e29dd8204139b6f76a48d18ec82e/users/0cfd18c7e5514c2f97e8b502005be024/roles/d0b9768f4d014766b9326387f2e11f34` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:24.363273 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7eef43-2a6c-4e0a-98af-e16563e901a7 admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:30:24.368482 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f26d61-fa8f-4f09-9a61-4d4d1f109d28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:30:24.369290 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f26d61-fa8f-4f09-9a61-4d4d1f109d28 admin admin] REQUEST_METHOD: `PUT` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:24.370088 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f26d61-fa8f-4f09-9a61-4d4d1f109d28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:24.370369 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f26d61-fa8f-4f09-9a61-4d4d1f109d28 admin admin] PATH_INFO: `/v3/projects/fdf681bb5f4d4840867692fc68e59fbf/users/7131b6de338c40d19ba55cc403096e13/roles/d009411496b8452b88e77ffb8db015a7` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:24.380865 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f26d61-fa8f-4f09-9a61-4d4d1f109d28 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fdf681bb5f4d4840867692fc68e59fbf, user_id=7131b6de338c40d19ba55cc403096e13, role_id=d009411496b8452b88e77ffb8db015a7)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:30:24.383304 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4c5f26-291a-47ea-9e20-be6bf23310ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=38d4e29dd8204139b6f76a48d18ec82e, user_id=0cfd18c7e5514c2f97e8b502005be024, role_id=d0b9768f4d014766b9326387f2e11f34)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:30:24.384207 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f26d61-fa8f-4f09-9a61-4d4d1f109d28 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:30:24.388178 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4c5f26-291a-47ea-9e20-be6bf23310ad admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:30:24.394550 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:30:24.400381 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1341/2678] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 01:30:24 2022] PUT /identity/v3/projects/fdf681bb5f4d4840867692fc68e59fbf/users/7131b6de338c40d19ba55cc403096e13/roles/d009411496b8452b88e77ffb8db015a7 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 01:30:24.414284 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74afc8c-cf83-429c-93e7-16910ce8c860 None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:24.414284 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74afc8c-cf83-429c-93e7-16910ce8c860 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:24.414284 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74afc8c-cf83-429c-93e7-16910ce8c860 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:24.439548 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-f74afc8c-cf83-429c-93e7-16910ce8c860 None None] MFA Rules not processed for user `7131b6de338c40d19ba55cc403096e13`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:30:24.440818 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-f74afc8c-cf83-429c-93e7-16910ce8c860 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:30:24.490903 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1338/2679] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 01:30:24 2022] PUT /identity/v3/projects/38d4e29dd8204139b6f76a48d18ec82e/users/0cfd18c7e5514c2f97e8b502005be024/roles/d0b9768f4d014766b9326387f2e11f34 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:30:24.503183 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aad68ac-7ebb-4f72-baa8-6de4a9036f6a None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:24.503731 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aad68ac-7ebb-4f72-baa8-6de4a9036f6a None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:24.504348 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aad68ac-7ebb-4f72-baa8-6de4a9036f6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:24.519972 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1342/2680] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:30:24 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 01:30:24.531408 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74afc8c-cf83-429c-93e7-16910ce8c860 None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:30:24.533040 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-f74afc8c-cf83-429c-93e7-16910ce8c860 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:30:24.538023 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-6aad68ac-7ebb-4f72-baa8-6de4a9036f6a None None] MFA Rules not processed for user `0cfd18c7e5514c2f97e8b502005be024`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:30:24.539217 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-6aad68ac-7ebb-4f72-baa8-6de4a9036f6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:30:24.543944 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:30:24.598604 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74afc8c-cf83-429c-93e7-16910ce8c860 None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:30:24.613656 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1339/2681] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:30:24 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:30:24.614551 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aad68ac-7ebb-4f72-baa8-6de4a9036f6a None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:30:24.673600 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a33981-2038-4569-9f47-7d12627d1ac5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5c8fc69c518412ba1aad61a9f4f558c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:30:24.674389 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a33981-2038-4569-9f47-7d12627d1ac5 service cinder] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:24.674727 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a33981-2038-4569-9f47-7d12627d1ac5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:24.674994 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a33981-2038-4569-9f47-7d12627d1ac5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:24.696751 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aad68ac-7ebb-4f72-baa8-6de4a9036f6a None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:30:24.703339 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cbcfa2-661b-4ec1-ba5d-eaf75704d158 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:30:24.703963 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cbcfa2-661b-4ec1-ba5d-eaf75704d158 admin admin] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:24.704250 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cbcfa2-661b-4ec1-ba5d-eaf75704d158 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:24.704530 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cbcfa2-661b-4ec1-ba5d-eaf75704d158 admin admin] PATH_INFO: `/v3/projects` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:24.705462 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cbcfa2-661b-4ec1-ba5d-eaf75704d158 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:30:24.707519 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cbcfa2-661b-4ec1-ba5d-eaf75704d158 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:30:24.751437 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=100 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:30:24.753920 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a33981-2038-4569-9f47-7d12627d1ac5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:30:24.755363 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a33981-2038-4569-9f47-7d12627d1ac5 service cinder] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:30:24.764717 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1343/2682] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:30:24 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 01:30:24.803562 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1340/2683] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 01:30:24 2022] POST /identity/v3/projects => generated 383 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:30:24.813081 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a33981-2038-4569-9f47-7d12627d1ac5 service cinder] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:30:24.826490 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a33981-2038-4569-9f47-7d12627d1ac5 service cinder] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:30:24.895510 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3633ad4-7773-455d-a190-6af2ba101a64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:30:24.896678 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3633ad4-7773-455d-a190-6af2ba101a64 admin admin] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:24.897013 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3633ad4-7773-455d-a190-6af2ba101a64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:24.897293 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3633ad4-7773-455d-a190-6af2ba101a64 admin admin] PATH_INFO: `/v3/users` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:24.897930 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3633ad4-7773-455d-a190-6af2ba101a64 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:30:24.899916 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3633ad4-7773-455d-a190-6af2ba101a64 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:30:24.981821 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1344/2684] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 01:30:24 2022] POST /identity/v3/users => generated 298 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:30:24.985878 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cbcfa2-661b-4ec1-ba5d-eaf75704d158 admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:30:24.997872 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cbcfa2-661b-4ec1-ba5d-eaf75704d158 admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:30:25.002910 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e859f0-d7a2-4b23-a3ce-8310b29e1b3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:30:25.003766 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e859f0-d7a2-4b23-a3ce-8310b29e1b3e admin admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:25.004069 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e859f0-d7a2-4b23-a3ce-8310b29e1b3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:25.004420 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e859f0-d7a2-4b23-a3ce-8310b29e1b3e admin admin] PATH_INFO: `/v3/roles` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:25.005105 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e859f0-d7a2-4b23-a3ce-8310b29e1b3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:30:25.006897 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e859f0-d7a2-4b23-a3ce-8310b29e1b3e admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:30:25.019155 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1341/2685] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:30:24 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:30:25.023281 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3633ad4-7773-455d-a190-6af2ba101a64 admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:30:25.034310 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3633ad4-7773-455d-a190-6af2ba101a64 admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:30:25.038963 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a45e23-8184-499c-9d3c-91bf512a2893 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:30:25.040048 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a45e23-8184-499c-9d3c-91bf512a2893 admin admin] REQUEST_METHOD: `PUT` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:25.040048 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a45e23-8184-499c-9d3c-91bf512a2893 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:25.040273 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a45e23-8184-499c-9d3c-91bf512a2893 admin admin] PATH_INFO: `/v3/projects/749026c7ea74462f8bed6cd0b4843fa6/users/6923c98eec7b4c47a21d40c7f471d706/roles/d009411496b8452b88e77ffb8db015a7` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:25.055644 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a45e23-8184-499c-9d3c-91bf512a2893 admin admin] RBAC: Authorizing `identity:create_grant(project_id=749026c7ea74462f8bed6cd0b4843fa6, user_id=6923c98eec7b4c47a21d40c7f471d706, role_id=d009411496b8452b88e77ffb8db015a7)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:30:25.058330 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a45e23-8184-499c-9d3c-91bf512a2893 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:30:25.153600 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1345/2686] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 01:30:25 2022] PUT /identity/v3/projects/749026c7ea74462f8bed6cd0b4843fa6/users/6923c98eec7b4c47a21d40c7f471d706/roles/d009411496b8452b88e77ffb8db015a7 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 01:30:25.157579 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e859f0-d7a2-4b23-a3ce-8310b29e1b3e admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:30:25.168248 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e859f0-d7a2-4b23-a3ce-8310b29e1b3e admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:30:25.227500 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983c47f2-c98b-4303-9aaa-631ed7eec6ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:30:25.228159 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983c47f2-c98b-4303-9aaa-631ed7eec6ef admin admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:25.228440 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983c47f2-c98b-4303-9aaa-631ed7eec6ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:25.228739 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983c47f2-c98b-4303-9aaa-631ed7eec6ef admin admin] PATH_INFO: `/v3/roles` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:25.230151 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983c47f2-c98b-4303-9aaa-631ed7eec6ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:30:25.231785 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983c47f2-c98b-4303-9aaa-631ed7eec6ef admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:30:25.242178 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1342/2687] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:30:25 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:30:25.246418 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a45e23-8184-499c-9d3c-91bf512a2893 admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:30:25.257043 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a45e23-8184-499c-9d3c-91bf512a2893 admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:30:25.262547 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc3cad3-a0dc-4221-b6fe-c905fb538e2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:30:25.263194 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc3cad3-a0dc-4221-b6fe-c905fb538e2e admin admin] REQUEST_METHOD: `PUT` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:25.263526 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc3cad3-a0dc-4221-b6fe-c905fb538e2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:25.263804 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc3cad3-a0dc-4221-b6fe-c905fb538e2e admin admin] PATH_INFO: `/v3/projects/749026c7ea74462f8bed6cd0b4843fa6/users/6923c98eec7b4c47a21d40c7f471d706/roles/d009411496b8452b88e77ffb8db015a7` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:25.266123 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc3cad3-a0dc-4221-b6fe-c905fb538e2e admin admin] RBAC: Authorizing `identity:create_grant(project_id=749026c7ea74462f8bed6cd0b4843fa6, user_id=6923c98eec7b4c47a21d40c7f471d706, role_id=d009411496b8452b88e77ffb8db015a7)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:30:25.268738 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc3cad3-a0dc-4221-b6fe-c905fb538e2e admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:30:25.279732 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1346/2688] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 01:30:25 2022] PUT /identity/v3/projects/749026c7ea74462f8bed6cd0b4843fa6/users/6923c98eec7b4c47a21d40c7f471d706/roles/d009411496b8452b88e77ffb8db015a7 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:30:25.289952 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24e1618-eed3-4fcb-b46f-9ff5ddfff206 None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:25.290248 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24e1618-eed3-4fcb-b46f-9ff5ddfff206 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:25.290530 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24e1618-eed3-4fcb-b46f-9ff5ddfff206 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:25.320417 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-a24e1618-eed3-4fcb-b46f-9ff5ddfff206 None None] MFA Rules not processed for user `6923c98eec7b4c47a21d40c7f471d706`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:30:25.321658 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-a24e1618-eed3-4fcb-b46f-9ff5ddfff206 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:30:25.325927 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:30:25.409135 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1343/2689] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:30:25 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:30:25.418061 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc3cad3-a0dc-4221-b6fe-c905fb538e2e admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:30:25.503910 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc3cad3-a0dc-4221-b6fe-c905fb538e2e admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:30:25.507561 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbec99ff-fb63-48a4-898b-42f3f71011bb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5c8fc69c518412ba1aad61a9f4f558c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:30:25.508152 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbec99ff-fb63-48a4-898b-42f3f71011bb service cinder] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:25.508431 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbec99ff-fb63-48a4-898b-42f3f71011bb service cinder] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:25.508704 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbec99ff-fb63-48a4-898b-42f3f71011bb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:25.564174 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbec99ff-fb63-48a4-898b-42f3f71011bb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:30:25.565645 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbec99ff-fb63-48a4-898b-42f3f71011bb service cinder] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:30:25.574831 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1347/2690] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:30:25 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:30:28.553595 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24e1618-eed3-4fcb-b46f-9ff5ddfff206 None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:30:28.574352 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24e1618-eed3-4fcb-b46f-9ff5ddfff206 None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:30:28.578734 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9729620b-62ac-4b4d-9285-62ab3b092b5d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5c8fc69c518412ba1aad61a9f4f558c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:30:28.579342 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9729620b-62ac-4b4d-9285-62ab3b092b5d service cinder] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:28.579856 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9729620b-62ac-4b4d-9285-62ab3b092b5d service cinder] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:28.580377 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9729620b-62ac-4b4d-9285-62ab3b092b5d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:28.655713 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9729620b-62ac-4b4d-9285-62ab3b092b5d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:30:28.657339 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9729620b-62ac-4b4d-9285-62ab3b092b5d service cinder] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:30:28.666667 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1344/2691] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:30:28 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:30:29.985569 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbec99ff-fb63-48a4-898b-42f3f71011bb service cinder] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:30:29.988225 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:30:30.092570 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbec99ff-fb63-48a4-898b-42f3f71011bb service cinder] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:30:30.097232 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23972762-8876-4114-86cc-ce87b8f2d8c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:30:30.102717 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23972762-8876-4114-86cc-ce87b8f2d8c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:30.102717 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23972762-8876-4114-86cc-ce87b8f2d8c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:30.102717 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23972762-8876-4114-86cc-ce87b8f2d8c7 admin admin] PATH_INFO: `/v3/users/c61b3e307a2b4a8e9f323cd836090a1b` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:30.102717 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23972762-8876-4114-86cc-ce87b8f2d8c7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c61b3e307a2b4a8e9f323cd836090a1b)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:30:30.102717 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23972762-8876-4114-86cc-ce87b8f2d8c7 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:30:30.129359 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9729620b-62ac-4b4d-9285-62ab3b092b5d service cinder] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:30:30.139839 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9729620b-62ac-4b4d-9285-62ab3b092b5d service cinder] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:30:30.143545 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cb0eea-cce0-4d14-9421-58b34bf1cad9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5c8fc69c518412ba1aad61a9f4f558c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:30:30.144149 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cb0eea-cce0-4d14-9421-58b34bf1cad9 service cinder] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:30.144437 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cb0eea-cce0-4d14-9421-58b34bf1cad9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:30.144715 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cb0eea-cce0-4d14-9421-58b34bf1cad9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:30.153826 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cb0eea-cce0-4d14-9421-58b34bf1cad9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:30:30.155315 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cb0eea-cce0-4d14-9421-58b34bf1cad9 service cinder] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:30:30.164938 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1345/2692] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:30:30 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 01:30:30.349203 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-23972762-8876-4114-86cc-ce87b8f2d8c7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c61b3e307a2b4a8e9f323cd836090a1b'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:30.361178 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-23972762-8876-4114-86cc-ce87b8f2d8c7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c61b3e307a2b4a8e9f323cd836090a1b'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:30.361907 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-23972762-8876-4114-86cc-ce87b8f2d8c7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c61b3e307a2b4a8e9f323cd836090a1b'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:30.369433 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-23972762-8876-4114-86cc-ce87b8f2d8c7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c61b3e307a2b4a8e9f323cd836090a1b'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:30.443545 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-23972762-8876-4114-86cc-ce87b8f2d8c7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c61b3e307a2b4a8e9f323cd836090a1b'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:30.449907 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1348/2693] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 01:30:29 2022] DELETE /identity/v3/users/c61b3e307a2b4a8e9f323cd836090a1b => generated 0 bytes in 466 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:30:30.455532 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18347449-7b60-4976-b04a-6d7770f6ba09 None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:30.455828 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18347449-7b60-4976-b04a-6d7770f6ba09 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:30.456144 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18347449-7b60-4976-b04a-6d7770f6ba09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:30.505711 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-18347449-7b60-4976-b04a-6d7770f6ba09 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:30:30.506940 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-18347449-7b60-4976-b04a-6d7770f6ba09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:30:30.536989 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:30:30.568519 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1346/2694] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:30:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:30:30.578465 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23972762-8876-4114-86cc-ce87b8f2d8c7 admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:30:30.580387 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-23972762-8876-4114-86cc-ce87b8f2d8c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:30:30.624356 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:30:30.648032 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23972762-8876-4114-86cc-ce87b8f2d8c7 admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:30:30.651734 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b837e4-6897-4179-b306-bd1a8590c83f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0817c3320d5d41d8bba0550c8ae8dd60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:30:30.652328 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b837e4-6897-4179-b306-bd1a8590c83f service neutron] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:30.652644 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b837e4-6897-4179-b306-bd1a8590c83f service neutron] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:30.652963 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b837e4-6897-4179-b306-bd1a8590c83f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:30.662106 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b837e4-6897-4179-b306-bd1a8590c83f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:30:30.663529 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b837e4-6897-4179-b306-bd1a8590c83f service neutron] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:30:30.673987 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1349/2695] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:30:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:30:31.148380 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18347449-7b60-4976-b04a-6d7770f6ba09 None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:30:31.150196 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-18347449-7b60-4976-b04a-6d7770f6ba09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:30:31.167845 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18347449-7b60-4976-b04a-6d7770f6ba09 None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:30:31.173518 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4269ce4b-7efa-4403-83a2-5114d6bd8f32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:30:31.174161 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4269ce4b-7efa-4403-83a2-5114d6bd8f32 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:31.174451 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4269ce4b-7efa-4403-83a2-5114d6bd8f32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:31.174734 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4269ce4b-7efa-4403-83a2-5114d6bd8f32 admin admin] PATH_INFO: `/v3/projects/9850fb6b650349f280454d8870c90e48` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:31.176040 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4269ce4b-7efa-4403-83a2-5114d6bd8f32 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9850fb6b650349f280454d8870c90e48)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:30:31.178529 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4269ce4b-7efa-4403-83a2-5114d6bd8f32 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:30:31.285002 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-4269ce4b-7efa-4403-83a2-5114d6bd8f32 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9850fb6b650349f280454d8870c90e48', 'request_id': 'req-4269ce4b-7efa-4403-83a2-5114d6bd8f32'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:31.290612 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-4269ce4b-7efa-4403-83a2-5114d6bd8f32 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9850fb6b650349f280454d8870c90e48', 'request_id': 'req-4269ce4b-7efa-4403-83a2-5114d6bd8f32'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:31.297854 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-4269ce4b-7efa-4403-83a2-5114d6bd8f32 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9850fb6b650349f280454d8870c90e48', 'request_id': 'req-4269ce4b-7efa-4403-83a2-5114d6bd8f32'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:31.381654 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-4269ce4b-7efa-4403-83a2-5114d6bd8f32 admin admin] The token cache is being invalidate because project 9850fb6b650349f280454d8870c90e48 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81749) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 01:30:31.381970 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-4269ce4b-7efa-4403-83a2-5114d6bd8f32 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:31.382598 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-4269ce4b-7efa-4403-83a2-5114d6bd8f32 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:31.384068 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1347/2696] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 01:30:31 2022] DELETE /identity/v3/projects/9850fb6b650349f280454d8870c90e48 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:30:31.390726 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b837e4-6897-4179-b306-bd1a8590c83f service neutron] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:30:31.400410 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b837e4-6897-4179-b306-bd1a8590c83f service neutron] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:30:31.464990 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155b22b5-b0a7-4ad6-83f5-c6bdf0cb0655 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:30:31.466283 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155b22b5-b0a7-4ad6-83f5-c6bdf0cb0655 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:31.466576 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155b22b5-b0a7-4ad6-83f5-c6bdf0cb0655 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:31.466851 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155b22b5-b0a7-4ad6-83f5-c6bdf0cb0655 admin admin] PATH_INFO: `/v3/users/7131b6de338c40d19ba55cc403096e13` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:31.467971 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155b22b5-b0a7-4ad6-83f5-c6bdf0cb0655 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7131b6de338c40d19ba55cc403096e13)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:30:31.469933 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155b22b5-b0a7-4ad6-83f5-c6bdf0cb0655 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:30:31.531977 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4269ce4b-7efa-4403-83a2-5114d6bd8f32 admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:30:31.533540 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-4269ce4b-7efa-4403-83a2-5114d6bd8f32 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:30:31.546953 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4269ce4b-7efa-4403-83a2-5114d6bd8f32 admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:30:31.552145 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6b2e84-9104-41a6-84ff-5ed3ae692887 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:30:31.561441 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6b2e84-9104-41a6-84ff-5ed3ae692887 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:31.561441 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6b2e84-9104-41a6-84ff-5ed3ae692887 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:31.561441 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6b2e84-9104-41a6-84ff-5ed3ae692887 admin admin] PATH_INFO: `/v3/users/7e18507e425e4d3bbcfb6fb0c34e086e` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:31.561441 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6b2e84-9104-41a6-84ff-5ed3ae692887 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7e18507e425e4d3bbcfb6fb0c34e086e)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:30:31.561441 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6b2e84-9104-41a6-84ff-5ed3ae692887 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:30:31.704255 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-155b22b5-b0a7-4ad6-83f5-c6bdf0cb0655 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7131b6de338c40d19ba55cc403096e13'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:31.716453 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-155b22b5-b0a7-4ad6-83f5-c6bdf0cb0655 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7131b6de338c40d19ba55cc403096e13'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:31.717221 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-155b22b5-b0a7-4ad6-83f5-c6bdf0cb0655 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7131b6de338c40d19ba55cc403096e13'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:31.724254 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-155b22b5-b0a7-4ad6-83f5-c6bdf0cb0655 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7131b6de338c40d19ba55cc403096e13'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:31.810686 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-155b22b5-b0a7-4ad6-83f5-c6bdf0cb0655 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7131b6de338c40d19ba55cc403096e13'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:31.817066 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1350/2697] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 01:30:31 2022] DELETE /identity/v3/users/7131b6de338c40d19ba55cc403096e13 => generated 0 bytes in 427 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:30:31.823768 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-6e6b2e84-9104-41a6-84ff-5ed3ae692887 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7e18507e425e4d3bbcfb6fb0c34e086e'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:31.895303 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-6e6b2e84-9104-41a6-84ff-5ed3ae692887 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7e18507e425e4d3bbcfb6fb0c34e086e'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:31.896157 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-6e6b2e84-9104-41a6-84ff-5ed3ae692887 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7e18507e425e4d3bbcfb6fb0c34e086e'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:31.908059 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-6e6b2e84-9104-41a6-84ff-5ed3ae692887 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7e18507e425e4d3bbcfb6fb0c34e086e'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:31.908780 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-6e6b2e84-9104-41a6-84ff-5ed3ae692887 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7e18507e425e4d3bbcfb6fb0c34e086e'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:31.926514 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1348/2698] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 01:30:31 2022] DELETE /identity/v3/users/7e18507e425e4d3bbcfb6fb0c34e086e => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 01:30:31.931109 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81460c67-203d-4225-893c-986c2cf79e8d None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:31.933641 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81460c67-203d-4225-893c-986c2cf79e8d None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:31.933641 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81460c67-203d-4225-893c-986c2cf79e8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:31.951950 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-81460c67-203d-4225-893c-986c2cf79e8d None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:30:31.953571 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-81460c67-203d-4225-893c-986c2cf79e8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:30:32.003579 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:30:32.026998 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1351/2699] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:30:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 01:30:32.038411 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6b2e84-9104-41a6-84ff-5ed3ae692887 admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:30:32.040116 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-6e6b2e84-9104-41a6-84ff-5ed3ae692887 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:30:32.098587 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:30:32.123515 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6b2e84-9104-41a6-84ff-5ed3ae692887 admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:30:32.127692 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e582e67-0c85-4458-9c3c-0c3e6ebbdfdf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0817c3320d5d41d8bba0550c8ae8dd60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:30:32.128569 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e582e67-0c85-4458-9c3c-0c3e6ebbdfdf service neutron] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:32.128979 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e582e67-0c85-4458-9c3c-0c3e6ebbdfdf service neutron] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:32.129356 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e582e67-0c85-4458-9c3c-0c3e6ebbdfdf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:32.140610 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e582e67-0c85-4458-9c3c-0c3e6ebbdfdf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:30:32.142423 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e582e67-0c85-4458-9c3c-0c3e6ebbdfdf service neutron] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:30:32.154996 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1349/2700] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:30:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:30:32.300304 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81460c67-203d-4225-893c-986c2cf79e8d None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:30:32.301734 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-81460c67-203d-4225-893c-986c2cf79e8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:30:32.314199 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81460c67-203d-4225-893c-986c2cf79e8d None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:30:32.319140 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f513c408-4d61-4cfe-8dfd-01cbe5aa5b78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:30:32.319773 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f513c408-4d61-4cfe-8dfd-01cbe5aa5b78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:32.320061 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f513c408-4d61-4cfe-8dfd-01cbe5aa5b78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:32.320335 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f513c408-4d61-4cfe-8dfd-01cbe5aa5b78 admin admin] PATH_INFO: `/v3/projects/fdf681bb5f4d4840867692fc68e59fbf` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:32.321579 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f513c408-4d61-4cfe-8dfd-01cbe5aa5b78 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fdf681bb5f4d4840867692fc68e59fbf)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:30:32.323496 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f513c408-4d61-4cfe-8dfd-01cbe5aa5b78 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:30:32.449923 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-f513c408-4d61-4cfe-8dfd-01cbe5aa5b78 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fdf681bb5f4d4840867692fc68e59fbf', 'request_id': 'req-f513c408-4d61-4cfe-8dfd-01cbe5aa5b78'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:32.455402 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-f513c408-4d61-4cfe-8dfd-01cbe5aa5b78 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fdf681bb5f4d4840867692fc68e59fbf', 'request_id': 'req-f513c408-4d61-4cfe-8dfd-01cbe5aa5b78'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:32.541027 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-f513c408-4d61-4cfe-8dfd-01cbe5aa5b78 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fdf681bb5f4d4840867692fc68e59fbf', 'request_id': 'req-f513c408-4d61-4cfe-8dfd-01cbe5aa5b78'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:32.554578 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-f513c408-4d61-4cfe-8dfd-01cbe5aa5b78 admin admin] The token cache is being invalidate because project fdf681bb5f4d4840867692fc68e59fbf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81750) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 01:30:32.554877 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-f513c408-4d61-4cfe-8dfd-01cbe5aa5b78 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:32.555553 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-f513c408-4d61-4cfe-8dfd-01cbe5aa5b78 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:32.560943 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1352/2701] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 01:30:32 2022] DELETE /identity/v3/projects/fdf681bb5f4d4840867692fc68e59fbf => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:30:32.563275 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fc21e7-619e-454e-a900-36a7958d9eba None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:32.563697 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fc21e7-619e-454e-a900-36a7958d9eba None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:32.564081 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fc21e7-619e-454e-a900-36a7958d9eba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:32.618226 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-b0fc21e7-619e-454e-a900-36a7958d9eba None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:30:32.619453 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-b0fc21e7-619e-454e-a900-36a7958d9eba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:30:32.682156 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1350/2702] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:30:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:30:32.690164 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1616e38-8d3a-4fc8-a988-6f4ef4ea2248 None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:32.690450 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1616e38-8d3a-4fc8-a988-6f4ef4ea2248 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:32.690729 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1616e38-8d3a-4fc8-a988-6f4ef4ea2248 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:32.716223 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-b1616e38-8d3a-4fc8-a988-6f4ef4ea2248 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:30:32.716223 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-b1616e38-8d3a-4fc8-a988-6f4ef4ea2248 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:30:32.721674 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1353/2703] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:30:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:30:32.725963 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fc21e7-619e-454e-a900-36a7958d9eba None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:30:32.735856 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fc21e7-619e-454e-a900-36a7958d9eba None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:30:32.740616 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6e9200-3296-436d-af16-ef0887b6d314 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:30:32.741299 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6e9200-3296-436d-af16-ef0887b6d314 admin admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:32.741580 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6e9200-3296-436d-af16-ef0887b6d314 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:32.741863 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6e9200-3296-436d-af16-ef0887b6d314 admin admin] PATH_INFO: `/v3/domains` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:32.742510 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6e9200-3296-436d-af16-ef0887b6d314 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:30:32.742820 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6e9200-3296-436d-af16-ef0887b6d314 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:30:32.744435 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6e9200-3296-436d-af16-ef0887b6d314 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:30:32.757021 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1351/2704] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Aug 31 01:30:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:30:32.761503 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1616e38-8d3a-4fc8-a988-6f4ef4ea2248 None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:30:32.772139 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1616e38-8d3a-4fc8-a988-6f4ef4ea2248 None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:30:32.776898 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ef5e57-b3f1-44d8-8d75-d20a60ddffcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:30:32.777500 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ef5e57-b3f1-44d8-8d75-d20a60ddffcc admin admin] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:32.777786 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ef5e57-b3f1-44d8-8d75-d20a60ddffcc admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:32.778696 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ef5e57-b3f1-44d8-8d75-d20a60ddffcc admin admin] PATH_INFO: `/v3/projects` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:32.779379 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ef5e57-b3f1-44d8-8d75-d20a60ddffcc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:30:32.781356 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ef5e57-b3f1-44d8-8d75-d20a60ddffcc admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:30:32.822725 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6e9200-3296-436d-af16-ef0887b6d314 admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:30:32.828182 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-ba6e9200-3296-436d-af16-ef0887b6d314 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:30:32.838661 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6e9200-3296-436d-af16-ef0887b6d314 admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:30:32.844417 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee5c88c-c161-45fe-a60e-c3edd7a20f43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:30:32.845092 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee5c88c-c161-45fe-a60e-c3edd7a20f43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:32.845379 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee5c88c-c161-45fe-a60e-c3edd7a20f43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:32.845660 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee5c88c-c161-45fe-a60e-c3edd7a20f43 admin admin] PATH_INFO: `/v3/projects/11fd3ccf6b67442a94afe88f399863aa` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:32.846827 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee5c88c-c161-45fe-a60e-c3edd7a20f43 admin admin] RBAC: Authorizing `identity:delete_project(project_id=11fd3ccf6b67442a94afe88f399863aa)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:30:32.848938 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee5c88c-c161-45fe-a60e-c3edd7a20f43 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:30:32.914565 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1354/2705] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 01:30:32 2022] POST /identity/v3/projects => generated 375 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 01:30:32.918669 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ef5e57-b3f1-44d8-8d75-d20a60ddffcc admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:30:32.980117 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ef5e57-b3f1-44d8-8d75-d20a60ddffcc admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:30:32.981811 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-7ee5c88c-c161-45fe-a60e-c3edd7a20f43 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '11fd3ccf6b67442a94afe88f399863aa', 'request_id': 'req-7ee5c88c-c161-45fe-a60e-c3edd7a20f43'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:32.987661 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-7ee5c88c-c161-45fe-a60e-c3edd7a20f43 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '11fd3ccf6b67442a94afe88f399863aa', 'request_id': 'req-7ee5c88c-c161-45fe-a60e-c3edd7a20f43'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:32.995941 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-7ee5c88c-c161-45fe-a60e-c3edd7a20f43 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '11fd3ccf6b67442a94afe88f399863aa', 'request_id': 'req-7ee5c88c-c161-45fe-a60e-c3edd7a20f43'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:33.038137 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:30:33.045293 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0969c2-581c-491f-83c0-ea685c0e5a77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:30:33.051210 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0969c2-581c-491f-83c0-ea685c0e5a77 admin admin] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:33.051210 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0969c2-581c-491f-83c0-ea685c0e5a77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:33.051210 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0969c2-581c-491f-83c0-ea685c0e5a77 admin admin] PATH_INFO: `/v3/users` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:33.051210 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0969c2-581c-491f-83c0-ea685c0e5a77 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:30:33.051210 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0969c2-581c-491f-83c0-ea685c0e5a77 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:30:33.066885 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-7ee5c88c-c161-45fe-a60e-c3edd7a20f43 admin admin] The token cache is being invalidate because project 11fd3ccf6b67442a94afe88f399863aa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81749) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 01:30:33.069053 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-7ee5c88c-c161-45fe-a60e-c3edd7a20f43 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:33.069053 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-7ee5c88c-c161-45fe-a60e-c3edd7a20f43 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:33.069720 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1352/2706] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 01:30:32 2022] DELETE /identity/v3/projects/11fd3ccf6b67442a94afe88f399863aa => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 01:30:33.073845 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee5c88c-c161-45fe-a60e-c3edd7a20f43 admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:30:33.075859 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-7ee5c88c-c161-45fe-a60e-c3edd7a20f43 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:30:33.088690 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee5c88c-c161-45fe-a60e-c3edd7a20f43 admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:30:33.093409 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f65923-dd0b-4545-9a8b-f8d78aeb2a69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:30:33.094039 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f65923-dd0b-4545-9a8b-f8d78aeb2a69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:33.094327 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f65923-dd0b-4545-9a8b-f8d78aeb2a69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:33.094603 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f65923-dd0b-4545-9a8b-f8d78aeb2a69 admin admin] PATH_INFO: `/v3/users/22287b3e49ed4b00a830b2449b03638d` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:33.095786 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f65923-dd0b-4545-9a8b-f8d78aeb2a69 admin admin] RBAC: Authorizing `identity:delete_user(user_id=22287b3e49ed4b00a830b2449b03638d)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:30:33.097762 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f65923-dd0b-4545-9a8b-f8d78aeb2a69 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:30:33.175687 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1355/2707] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 01:30:32 2022] POST /identity/v3/users => generated 288 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 01:30:33.181471 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0969c2-581c-491f-83c0-ea685c0e5a77 admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:30:33.183133 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-5c0969c2-581c-491f-83c0-ea685c0e5a77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:30:33.195545 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0969c2-581c-491f-83c0-ea685c0e5a77 admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:30:33.204944 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6927aa-101d-4820-906a-fff727239cef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:30:33.207191 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6927aa-101d-4820-906a-fff727239cef admin admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:33.207191 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6927aa-101d-4820-906a-fff727239cef admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:33.207191 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6927aa-101d-4820-906a-fff727239cef admin admin] PATH_INFO: `/v3/roles` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:33.207191 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6927aa-101d-4820-906a-fff727239cef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:30:33.208904 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6927aa-101d-4820-906a-fff727239cef admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:30:33.219259 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1356/2708] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:30:33 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 01:30:33.223367 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6927aa-101d-4820-906a-fff727239cef admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:30:33.235935 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6927aa-101d-4820-906a-fff727239cef admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:30:33.238228 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bcc971-1f8d-4049-a8f5-ad941b35dad7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:30:33.238833 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bcc971-1f8d-4049-a8f5-ad941b35dad7 admin admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:33.239118 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bcc971-1f8d-4049-a8f5-ad941b35dad7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:33.239398 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bcc971-1f8d-4049-a8f5-ad941b35dad7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:33.240112 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bcc971-1f8d-4049-a8f5-ad941b35dad7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:30:33.241699 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bcc971-1f8d-4049-a8f5-ad941b35dad7 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:30:33.253143 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1357/2709] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:30:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 01:30:33.257263 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bcc971-1f8d-4049-a8f5-ad941b35dad7 admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:30:33.267312 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bcc971-1f8d-4049-a8f5-ad941b35dad7 admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:30:33.271978 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23b66dc-9833-406b-892e-fe3fb18ae57e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:30:33.272595 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23b66dc-9833-406b-892e-fe3fb18ae57e admin admin] REQUEST_METHOD: `PUT` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:33.273285 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23b66dc-9833-406b-892e-fe3fb18ae57e admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:33.273564 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23b66dc-9833-406b-892e-fe3fb18ae57e admin admin] PATH_INFO: `/v3/projects/541dad74c8a7404f969a872cc18e4dd8/users/0c58e96ca245457aadcb6ee9caac5b30/roles/d0b9768f4d014766b9326387f2e11f34` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:33.288706 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23b66dc-9833-406b-892e-fe3fb18ae57e admin admin] RBAC: Authorizing `identity:create_grant(project_id=541dad74c8a7404f969a872cc18e4dd8, user_id=0c58e96ca245457aadcb6ee9caac5b30, role_id=d0b9768f4d014766b9326387f2e11f34)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:30:33.291406 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23b66dc-9833-406b-892e-fe3fb18ae57e admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:30:33.334223 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-65f65923-dd0b-4545-9a8b-f8d78aeb2a69 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '22287b3e49ed4b00a830b2449b03638d'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:33.380483 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1358/2710] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 01:30:33 2022] PUT /identity/v3/projects/541dad74c8a7404f969a872cc18e4dd8/users/0c58e96ca245457aadcb6ee9caac5b30/roles/d0b9768f4d014766b9326387f2e11f34 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 01:30:33.392265 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196b062d-8f0c-4e9c-964e-1f0a8de8a039 None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:33.392546 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196b062d-8f0c-4e9c-964e-1f0a8de8a039 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:33.392871 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196b062d-8f0c-4e9c-964e-1f0a8de8a039 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:33.425156 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-196b062d-8f0c-4e9c-964e-1f0a8de8a039 None None] MFA Rules not processed for user `0c58e96ca245457aadcb6ee9caac5b30`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:30:33.426765 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-196b062d-8f0c-4e9c-964e-1f0a8de8a039 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:30:33.427106 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-65f65923-dd0b-4545-9a8b-f8d78aeb2a69 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '22287b3e49ed4b00a830b2449b03638d'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:33.427851 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-65f65923-dd0b-4545-9a8b-f8d78aeb2a69 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '22287b3e49ed4b00a830b2449b03638d'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:33.436843 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-65f65923-dd0b-4545-9a8b-f8d78aeb2a69 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '22287b3e49ed4b00a830b2449b03638d'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:33.437644 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-65f65923-dd0b-4545-9a8b-f8d78aeb2a69 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '22287b3e49ed4b00a830b2449b03638d'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:33.462493 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1353/2711] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 01:30:33 2022] DELETE /identity/v3/users/22287b3e49ed4b00a830b2449b03638d => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 01:30:33.518537 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1359/2712] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:30:33 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 01:30:33.523251 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f65923-dd0b-4545-9a8b-f8d78aeb2a69 admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:30:33.524866 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-65f65923-dd0b-4545-9a8b-f8d78aeb2a69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:30:33.576700 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:30:33.601197 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f65923-dd0b-4545-9a8b-f8d78aeb2a69 admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:30:33.608502 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5c3014-d4fe-4f89-8505-34dbe3ad45ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:30:33.609336 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5c3014-d4fe-4f89-8505-34dbe3ad45ec admin admin] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:33.609638 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5c3014-d4fe-4f89-8505-34dbe3ad45ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:33.610380 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5c3014-d4fe-4f89-8505-34dbe3ad45ec admin admin] PATH_INFO: `/v3/projects` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:33.611064 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5c3014-d4fe-4f89-8505-34dbe3ad45ec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:30:33.615041 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5c3014-d4fe-4f89-8505-34dbe3ad45ec admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:30:33.715394 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1354/2713] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 01:30:33 2022] POST /identity/v3/projects => generated 377 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 01:30:33.719391 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196b062d-8f0c-4e9c-964e-1f0a8de8a039 None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:30:33.792940 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196b062d-8f0c-4e9c-964e-1f0a8de8a039 None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:30:33.849932 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655f414c-abd9-40a7-81e5-7e5a6bd9bc79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:30:33.850566 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655f414c-abd9-40a7-81e5-7e5a6bd9bc79 admin admin] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:33.850842 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655f414c-abd9-40a7-81e5-7e5a6bd9bc79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:33.851122 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655f414c-abd9-40a7-81e5-7e5a6bd9bc79 admin admin] PATH_INFO: `/v3/users` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:33.853421 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655f414c-abd9-40a7-81e5-7e5a6bd9bc79 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:30:33.857435 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655f414c-abd9-40a7-81e5-7e5a6bd9bc79 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:30:33.931231 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5c3014-d4fe-4f89-8505-34dbe3ad45ec admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:30:33.936048 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-df5c3014-d4fe-4f89-8505-34dbe3ad45ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:30:33.947375 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5c3014-d4fe-4f89-8505-34dbe3ad45ec admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:30:33.957982 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1360/2714] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 01:30:33 2022] POST /identity/v3/users => generated 289 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:30:33.958546 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655f414c-abd9-40a7-81e5-7e5a6bd9bc79 admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:30:33.962415 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fdafa3-6009-420d-a156-6cd8ec2bc678 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:30:33.963047 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fdafa3-6009-420d-a156-6cd8ec2bc678 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:33.963342 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fdafa3-6009-420d-a156-6cd8ec2bc678 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:33.963676 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fdafa3-6009-420d-a156-6cd8ec2bc678 admin admin] PATH_INFO: `/v3/projects/2b562496d2a04f419921602e5a515919` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:33.965260 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fdafa3-6009-420d-a156-6cd8ec2bc678 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2b562496d2a04f419921602e5a515919)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:30:33.967979 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fdafa3-6009-420d-a156-6cd8ec2bc678 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:30:33.969493 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655f414c-abd9-40a7-81e5-7e5a6bd9bc79 admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:30:33.975031 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fb01ec-b05a-49e2-b7a0-87c03d464e89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:30:33.975648 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fb01ec-b05a-49e2-b7a0-87c03d464e89 admin admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:33.982014 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fb01ec-b05a-49e2-b7a0-87c03d464e89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:33.982014 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fb01ec-b05a-49e2-b7a0-87c03d464e89 admin admin] PATH_INFO: `/v3/roles` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:33.982014 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fb01ec-b05a-49e2-b7a0-87c03d464e89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:30:33.982014 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fb01ec-b05a-49e2-b7a0-87c03d464e89 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:30:33.995362 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1361/2715] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:30:33 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 01:30:34.000265 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fb01ec-b05a-49e2-b7a0-87c03d464e89 admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:30:34.010183 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fb01ec-b05a-49e2-b7a0-87c03d464e89 admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:30:34.016641 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487c5546-d80d-45f5-9da3-5734a44e9503 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:30:34.017270 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487c5546-d80d-45f5-9da3-5734a44e9503 admin admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:34.017546 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487c5546-d80d-45f5-9da3-5734a44e9503 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:34.017814 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487c5546-d80d-45f5-9da3-5734a44e9503 admin admin] PATH_INFO: `/v3/roles` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:34.018446 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487c5546-d80d-45f5-9da3-5734a44e9503 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:30:34.020057 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487c5546-d80d-45f5-9da3-5734a44e9503 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:30:34.030480 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1362/2716] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:30:33 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:30:34.034917 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487c5546-d80d-45f5-9da3-5734a44e9503 admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:30:34.044406 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487c5546-d80d-45f5-9da3-5734a44e9503 admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:30:34.049317 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fefe6c-1e25-4424-bdb1-42900e90dd3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:30:34.049955 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fefe6c-1e25-4424-bdb1-42900e90dd3d admin admin] REQUEST_METHOD: `PUT` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:34.050236 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fefe6c-1e25-4424-bdb1-42900e90dd3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:34.050511 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fefe6c-1e25-4424-bdb1-42900e90dd3d admin admin] PATH_INFO: `/v3/projects/882072fe98cc4ca09c0f8859419af6b8/users/2fefd987a5da4fa59cb6464aa30a2470/roles/d0b9768f4d014766b9326387f2e11f34` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:34.066115 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fefe6c-1e25-4424-bdb1-42900e90dd3d admin admin] RBAC: Authorizing `identity:create_grant(project_id=882072fe98cc4ca09c0f8859419af6b8, user_id=2fefd987a5da4fa59cb6464aa30a2470, role_id=d0b9768f4d014766b9326387f2e11f34)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:30:34.068793 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fefe6c-1e25-4424-bdb1-42900e90dd3d admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:30:34.154395 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1363/2717] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 01:30:34 2022] PUT /identity/v3/projects/882072fe98cc4ca09c0f8859419af6b8/users/2fefd987a5da4fa59cb6464aa30a2470/roles/d0b9768f4d014766b9326387f2e11f34 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 01:30:34.162040 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-d7fdafa3-6009-420d-a156-6cd8ec2bc678 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2b562496d2a04f419921602e5a515919', 'request_id': 'req-d7fdafa3-6009-420d-a156-6cd8ec2bc678'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:34.165402 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec1c6e5-5d5b-492d-ac18-d5c1a6b664e2 None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:34.165794 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec1c6e5-5d5b-492d-ac18-d5c1a6b664e2 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:34.166143 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec1c6e5-5d5b-492d-ac18-d5c1a6b664e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:34.167654 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-d7fdafa3-6009-420d-a156-6cd8ec2bc678 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2b562496d2a04f419921602e5a515919', 'request_id': 'req-d7fdafa3-6009-420d-a156-6cd8ec2bc678'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:34.174884 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-d7fdafa3-6009-420d-a156-6cd8ec2bc678 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2b562496d2a04f419921602e5a515919', 'request_id': 'req-d7fdafa3-6009-420d-a156-6cd8ec2bc678'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:34.194358 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-5ec1c6e5-5d5b-492d-ac18-d5c1a6b664e2 None None] MFA Rules not processed for user `2fefd987a5da4fa59cb6464aa30a2470`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:30:34.195575 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-5ec1c6e5-5d5b-492d-ac18-d5c1a6b664e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:30:34.238545 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-d7fdafa3-6009-420d-a156-6cd8ec2bc678 admin admin] The token cache is being invalidate because project 2b562496d2a04f419921602e5a515919 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81749) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 01:30:34.238935 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-d7fdafa3-6009-420d-a156-6cd8ec2bc678 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:34.239652 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-d7fdafa3-6009-420d-a156-6cd8ec2bc678 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:34.241284 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1355/2718] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 01:30:33 2022] DELETE /identity/v3/projects/2b562496d2a04f419921602e5a515919 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 01:30:34.246204 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a1bdf0-0f4f-4a61-b91f-1dfbbf046759 None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:34.246597 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a1bdf0-0f4f-4a61-b91f-1dfbbf046759 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:34.247453 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a1bdf0-0f4f-4a61-b91f-1dfbbf046759 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:34.263257 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1364/2719] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:30:34 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:30:34.266970 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-e3a1bdf0-0f4f-4a61-b91f-1dfbbf046759 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:30:34.267725 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec1c6e5-5d5b-492d-ac18-d5c1a6b664e2 None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:30:34.268386 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-e3a1bdf0-0f4f-4a61-b91f-1dfbbf046759 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:30:34.269018 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-5ec1c6e5-5d5b-492d-ac18-d5c1a6b664e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:30:34.328927 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1356/2720] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:30:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 01:30:34.337463 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec1c6e5-5d5b-492d-ac18-d5c1a6b664e2 None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:30:34.337671 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00756a6-4e28-4eb7-902f-4a3fecffb0fb None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:34.338109 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00756a6-4e28-4eb7-902f-4a3fecffb0fb None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:34.338219 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00756a6-4e28-4eb7-902f-4a3fecffb0fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:34.344988 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93c9eaa-0441-4a33-a24c-3112042df8cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:30:34.345629 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93c9eaa-0441-4a33-a24c-3112042df8cb admin admin] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:34.345914 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93c9eaa-0441-4a33-a24c-3112042df8cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:34.346194 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93c9eaa-0441-4a33-a24c-3112042df8cb admin admin] PATH_INFO: `/v3/projects` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:34.346882 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93c9eaa-0441-4a33-a24c-3112042df8cb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:30:34.349013 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93c9eaa-0441-4a33-a24c-3112042df8cb admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:30:34.359714 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-b00756a6-4e28-4eb7-902f-4a3fecffb0fb None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:30:34.360991 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-b00756a6-4e28-4eb7-902f-4a3fecffb0fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:30:34.369121 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1357/2721] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:30:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 01:30:34.373967 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00756a6-4e28-4eb7-902f-4a3fecffb0fb None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:30:34.383420 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00756a6-4e28-4eb7-902f-4a3fecffb0fb None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:30:34.388076 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41159490-bd73-4cd2-99b7-8adb66062d2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:30:34.388749 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41159490-bd73-4cd2-99b7-8adb66062d2d admin admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:34.389131 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41159490-bd73-4cd2-99b7-8adb66062d2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:34.389406 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41159490-bd73-4cd2-99b7-8adb66062d2d admin admin] PATH_INFO: `/v3/domains` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:34.390093 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41159490-bd73-4cd2-99b7-8adb66062d2d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:30:34.390403 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41159490-bd73-4cd2-99b7-8adb66062d2d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:30:34.392010 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41159490-bd73-4cd2-99b7-8adb66062d2d admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:30:34.403343 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1358/2722] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Aug 31 01:30:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 01:30:34.407344 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41159490-bd73-4cd2-99b7-8adb66062d2d admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:30:34.417392 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41159490-bd73-4cd2-99b7-8adb66062d2d admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:30:34.423052 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bee4fa-c927-4b79-9f26-233350805c3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:30:34.423642 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bee4fa-c927-4b79-9f26-233350805c3d admin admin] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:34.423974 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bee4fa-c927-4b79-9f26-233350805c3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:34.424466 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bee4fa-c927-4b79-9f26-233350805c3d admin admin] PATH_INFO: `/v3/projects` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:34.425132 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bee4fa-c927-4b79-9f26-233350805c3d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:30:34.427058 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bee4fa-c927-4b79-9f26-233350805c3d admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:30:34.451125 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1365/2723] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 01:30:34 2022] POST /identity/v3/projects => generated 375 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:30:34.456831 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93c9eaa-0441-4a33-a24c-3112042df8cb admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:30:34.480908 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:30:34.517166 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93c9eaa-0441-4a33-a24c-3112042df8cb admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:30:34.522230 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3335e73c-adf8-40bd-bd95-a644f185170a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:30:34.522826 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3335e73c-adf8-40bd-bd95-a644f185170a admin admin] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:34.523113 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3335e73c-adf8-40bd-bd95-a644f185170a admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:34.523392 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3335e73c-adf8-40bd-bd95-a644f185170a admin admin] PATH_INFO: `/v3/users` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:34.524065 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3335e73c-adf8-40bd-bd95-a644f185170a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:30:34.525715 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1359/2724] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 01:30:34 2022] POST /identity/v3/projects => generated 381 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 01:30:34.525995 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3335e73c-adf8-40bd-bd95-a644f185170a admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:30:34.530336 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=81,keystone:DELETE=16,keystone:INSERT=3 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:30:34.533001 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bee4fa-c927-4b79-9f26-233350805c3d admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:30:34.594406 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bee4fa-c927-4b79-9f26-233350805c3d admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:30:34.599063 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724364c7-6b7b-45b2-9517-81e7161493eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:30:34.603706 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724364c7-6b7b-45b2-9517-81e7161493eb admin admin] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:34.603706 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724364c7-6b7b-45b2-9517-81e7161493eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:34.603706 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724364c7-6b7b-45b2-9517-81e7161493eb admin admin] PATH_INFO: `/v3/users` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:34.603706 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724364c7-6b7b-45b2-9517-81e7161493eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:30:34.603706 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724364c7-6b7b-45b2-9517-81e7161493eb admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:30:34.632876 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1366/2725] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 01:30:34 2022] POST /identity/v3/users => generated 294 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 01:30:34.637143 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3335e73c-adf8-40bd-bd95-a644f185170a admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:30:34.649140 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3335e73c-adf8-40bd-bd95-a644f185170a admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:30:34.653845 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f384efe7-cda7-40b5-87a6-8c2d9d838e60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:30:34.654951 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f384efe7-cda7-40b5-87a6-8c2d9d838e60 admin admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:34.655234 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f384efe7-cda7-40b5-87a6-8c2d9d838e60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:34.655508 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f384efe7-cda7-40b5-87a6-8c2d9d838e60 admin admin] PATH_INFO: `/v3/roles` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:34.656138 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f384efe7-cda7-40b5-87a6-8c2d9d838e60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:30:34.657899 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f384efe7-cda7-40b5-87a6-8c2d9d838e60 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:30:34.667993 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1367/2726] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:30:34 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 01:30:34.672519 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f384efe7-cda7-40b5-87a6-8c2d9d838e60 admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:30:34.681887 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f384efe7-cda7-40b5-87a6-8c2d9d838e60 admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:30:34.686597 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935b11ec-13f9-4adb-bb20-3969c09e8c8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:30:34.687207 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935b11ec-13f9-4adb-bb20-3969c09e8c8e admin admin] REQUEST_METHOD: `PUT` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:34.687492 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935b11ec-13f9-4adb-bb20-3969c09e8c8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:34.687772 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935b11ec-13f9-4adb-bb20-3969c09e8c8e admin admin] PATH_INFO: `/v3/projects/ea06efa087214c1eb593aae0cdf95154/users/82db35e7245240eb8e4af8b288fc42ea/roles/d009411496b8452b88e77ffb8db015a7` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:34.696070 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1360/2727] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 01:30:34 2022] POST /identity/v3/users => generated 291 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 01:30:34.699955 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724364c7-6b7b-45b2-9517-81e7161493eb admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:30:34.703674 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935b11ec-13f9-4adb-bb20-3969c09e8c8e admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea06efa087214c1eb593aae0cdf95154, user_id=82db35e7245240eb8e4af8b288fc42ea, role_id=d009411496b8452b88e77ffb8db015a7)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:30:34.707344 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935b11ec-13f9-4adb-bb20-3969c09e8c8e admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:30:34.710453 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724364c7-6b7b-45b2-9517-81e7161493eb admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:30:34.711211 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:30:34.715298 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ace081-e007-42bc-b458-3c8e78a92984 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:30:34.715896 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ace081-e007-42bc-b458-3c8e78a92984 admin admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:34.716185 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ace081-e007-42bc-b458-3c8e78a92984 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:34.716467 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ace081-e007-42bc-b458-3c8e78a92984 admin admin] PATH_INFO: `/v3/roles` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:34.717708 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ace081-e007-42bc-b458-3c8e78a92984 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:30:34.719333 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ace081-e007-42bc-b458-3c8e78a92984 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:30:34.729283 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1361/2728] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:30:34 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 01:30:34.733744 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ace081-e007-42bc-b458-3c8e78a92984 admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:30:34.743180 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ace081-e007-42bc-b458-3c8e78a92984 admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:30:34.748878 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc7ddfb-aa98-42e9-aba8-b7f688357560 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:30:34.749466 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc7ddfb-aa98-42e9-aba8-b7f688357560 admin admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:34.749789 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc7ddfb-aa98-42e9-aba8-b7f688357560 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:34.750111 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc7ddfb-aa98-42e9-aba8-b7f688357560 admin admin] PATH_INFO: `/v3/roles` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:34.750741 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc7ddfb-aa98-42e9-aba8-b7f688357560 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:30:34.752358 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc7ddfb-aa98-42e9-aba8-b7f688357560 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:30:34.764016 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1362/2729] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:30:34 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 01:30:34.767981 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc7ddfb-aa98-42e9-aba8-b7f688357560 admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:30:34.777135 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc7ddfb-aa98-42e9-aba8-b7f688357560 admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:30:34.784342 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8fa92a-94c3-4cc6-833d-9a098ed9c4ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:30:34.784342 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8fa92a-94c3-4cc6-833d-9a098ed9c4ca admin admin] REQUEST_METHOD: `PUT` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:34.784342 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8fa92a-94c3-4cc6-833d-9a098ed9c4ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:34.784342 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8fa92a-94c3-4cc6-833d-9a098ed9c4ca admin admin] PATH_INFO: `/v3/projects/8c7016c9f3424dd385816e4d25b0d02d/users/1caf2edea599429c96cdc90b240bcb78/roles/d0b9768f4d014766b9326387f2e11f34` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:34.798226 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8fa92a-94c3-4cc6-833d-9a098ed9c4ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c7016c9f3424dd385816e4d25b0d02d, user_id=1caf2edea599429c96cdc90b240bcb78, role_id=d0b9768f4d014766b9326387f2e11f34)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:30:34.800877 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8fa92a-94c3-4cc6-833d-9a098ed9c4ca admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:30:34.809394 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1368/2730] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 01:30:34 2022] PUT /identity/v3/projects/ea06efa087214c1eb593aae0cdf95154/users/82db35e7245240eb8e4af8b288fc42ea/roles/d009411496b8452b88e77ffb8db015a7 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 01:30:34.813473 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935b11ec-13f9-4adb-bb20-3969c09e8c8e admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:30:34.874417 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935b11ec-13f9-4adb-bb20-3969c09e8c8e admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:30:34.879247 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee31fd2-8fc3-4c87-ad01-373d4bedb5ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:30:34.879848 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee31fd2-8fc3-4c87-ad01-373d4bedb5ea admin admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:34.880123 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee31fd2-8fc3-4c87-ad01-373d4bedb5ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:34.880417 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee31fd2-8fc3-4c87-ad01-373d4bedb5ea admin admin] PATH_INFO: `/v3/roles` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:34.881126 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee31fd2-8fc3-4c87-ad01-373d4bedb5ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:30:34.882693 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee31fd2-8fc3-4c87-ad01-373d4bedb5ea admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:30:34.892381 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1369/2731] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:30:34 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 01:30:34.896304 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee31fd2-8fc3-4c87-ad01-373d4bedb5ea admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:30:34.900397 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1363/2732] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 01:30:34 2022] PUT /identity/v3/projects/8c7016c9f3424dd385816e4d25b0d02d/users/1caf2edea599429c96cdc90b240bcb78/roles/d0b9768f4d014766b9326387f2e11f34 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 01:30:34.911029 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee31fd2-8fc3-4c87-ad01-373d4bedb5ea admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:30:34.916426 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753b7934-24cd-4ecb-a8b5-e8663a81c31b None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:34.916426 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753b7934-24cd-4ecb-a8b5-e8663a81c31b None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:34.916426 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753b7934-24cd-4ecb-a8b5-e8663a81c31b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:34.922894 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:30:34.949360 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-753b7934-24cd-4ecb-a8b5-e8663a81c31b None None] MFA Rules not processed for user `1caf2edea599429c96cdc90b240bcb78`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:30:34.950642 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-753b7934-24cd-4ecb-a8b5-e8663a81c31b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:30:34.969114 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:30:34.986797 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288d99b3-39cb-4bb6-8910-64a63b7ef04b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:30:34.987064 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288d99b3-39cb-4bb6-8910-64a63b7ef04b admin admin] REQUEST_METHOD: `PUT` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:34.987347 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288d99b3-39cb-4bb6-8910-64a63b7ef04b admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:34.987625 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288d99b3-39cb-4bb6-8910-64a63b7ef04b admin admin] PATH_INFO: `/v3/projects/ea06efa087214c1eb593aae0cdf95154/users/82db35e7245240eb8e4af8b288fc42ea/roles/d009411496b8452b88e77ffb8db015a7` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:34.989892 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288d99b3-39cb-4bb6-8910-64a63b7ef04b admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea06efa087214c1eb593aae0cdf95154, user_id=82db35e7245240eb8e4af8b288fc42ea, role_id=d009411496b8452b88e77ffb8db015a7)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:30:34.994251 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288d99b3-39cb-4bb6-8910-64a63b7ef04b admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:30:35.005045 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1370/2733] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 01:30:34 2022] PUT /identity/v3/projects/ea06efa087214c1eb593aae0cdf95154/users/82db35e7245240eb8e4af8b288fc42ea/roles/d009411496b8452b88e77ffb8db015a7 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 01:30:35.005854 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288d99b3-39cb-4bb6-8910-64a63b7ef04b admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:30:35.007424 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-288d99b3-39cb-4bb6-8910-64a63b7ef04b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:30:35.030398 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1364/2734] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:30:34 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:30:35.032112 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993842f0-da8b-4159-b8c0-fab0eece218c None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:35.032602 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993842f0-da8b-4159-b8c0-fab0eece218c None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:35.033091 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993842f0-da8b-4159-b8c0-fab0eece218c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:35.061398 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-993842f0-da8b-4159-b8c0-fab0eece218c None None] MFA Rules not processed for user `82db35e7245240eb8e4af8b288fc42ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:30:35.062674 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-993842f0-da8b-4159-b8c0-fab0eece218c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:30:35.081908 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288d99b3-39cb-4bb6-8910-64a63b7ef04b admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:30:35.086743 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ae9ad0-54fe-40cf-aeaa-b46867adb2f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:30:35.087371 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ae9ad0-54fe-40cf-aeaa-b46867adb2f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:35.087652 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ae9ad0-54fe-40cf-aeaa-b46867adb2f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:35.087928 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ae9ad0-54fe-40cf-aeaa-b46867adb2f1 admin admin] PATH_INFO: `/v3/users/69cb6d9aff9d4766903e728218f905e3` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:35.089342 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ae9ad0-54fe-40cf-aeaa-b46867adb2f1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=69cb6d9aff9d4766903e728218f905e3)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:30:35.092871 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ae9ad0-54fe-40cf-aeaa-b46867adb2f1 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:30:35.136015 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1365/2735] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:30:35 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:30:35.137070 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993842f0-da8b-4159-b8c0-fab0eece218c None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:30:35.148535 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993842f0-da8b-4159-b8c0-fab0eece218c None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:30:35.153450 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48ad090-0f18-44a0-8b51-cdbe91d5503a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:30:35.154689 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48ad090-0f18-44a0-8b51-cdbe91d5503a admin admin] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:35.154977 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48ad090-0f18-44a0-8b51-cdbe91d5503a admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:35.155257 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48ad090-0f18-44a0-8b51-cdbe91d5503a admin admin] PATH_INFO: `/v3/projects` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:35.155890 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48ad090-0f18-44a0-8b51-cdbe91d5503a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:30:35.157998 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48ad090-0f18-44a0-8b51-cdbe91d5503a admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:30:35.279976 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1366/2736] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 01:30:35 2022] POST /identity/v3/projects => generated 381 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:30:35.280861 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48ad090-0f18-44a0-8b51-cdbe91d5503a admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:30:35.282524 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-f48ad090-0f18-44a0-8b51-cdbe91d5503a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:30:35.314872 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-e2ae9ad0-54fe-40cf-aeaa-b46867adb2f1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '69cb6d9aff9d4766903e728218f905e3'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:35.327938 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-e2ae9ad0-54fe-40cf-aeaa-b46867adb2f1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '69cb6d9aff9d4766903e728218f905e3'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:35.328637 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-e2ae9ad0-54fe-40cf-aeaa-b46867adb2f1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '69cb6d9aff9d4766903e728218f905e3'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:35.336456 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-e2ae9ad0-54fe-40cf-aeaa-b46867adb2f1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '69cb6d9aff9d4766903e728218f905e3'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:35.340322 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48ad090-0f18-44a0-8b51-cdbe91d5503a admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:30:35.344063 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5fd51e-8545-49a3-b928-39e7df2a4b54 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5c8fc69c518412ba1aad61a9f4f558c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:30:35.344792 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5fd51e-8545-49a3-b928-39e7df2a4b54 service cinder] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:35.345090 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5fd51e-8545-49a3-b928-39e7df2a4b54 service cinder] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:35.345371 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5fd51e-8545-49a3-b928-39e7df2a4b54 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:35.347111 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-bd5fd51e-8545-49a3-b928-39e7df2a4b54 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:30:35.356161 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:30:35.397736 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-e2ae9ad0-54fe-40cf-aeaa-b46867adb2f1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '69cb6d9aff9d4766903e728218f905e3'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:35.403967 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1371/2737] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 01:30:35 2022] DELETE /identity/v3/users/69cb6d9aff9d4766903e728218f905e3 => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:30:35.407139 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ae9ad0-54fe-40cf-aeaa-b46867adb2f1 admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:30:35.407139 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-e2ae9ad0-54fe-40cf-aeaa-b46867adb2f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:30:35.407350 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5fd51e-8545-49a3-b928-39e7df2a4b54 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:30:35.407856 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5fd51e-8545-49a3-b928-39e7df2a4b54 service cinder] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:30:35.418207 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1367/2738] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:30:35 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:30:35.419527 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894fd285-1c84-4542-9bc4-d1a71b9799b6 None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:35.419820 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894fd285-1c84-4542-9bc4-d1a71b9799b6 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:35.420103 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894fd285-1c84-4542-9bc4-d1a71b9799b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:35.441727 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-894fd285-1c84-4542-9bc4-d1a71b9799b6 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:30:35.443010 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-894fd285-1c84-4542-9bc4-d1a71b9799b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:30:35.479839 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:30:35.483845 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ae9ad0-54fe-40cf-aeaa-b46867adb2f1 admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:30:35.489047 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bba6e87-d4af-4ee9-9b18-9e7dcc5c8eaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:30:35.491792 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bba6e87-d4af-4ee9-9b18-9e7dcc5c8eaf admin admin] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:35.491792 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bba6e87-d4af-4ee9-9b18-9e7dcc5c8eaf admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:35.491792 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bba6e87-d4af-4ee9-9b18-9e7dcc5c8eaf admin admin] PATH_INFO: `/v3/users` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:35.491792 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bba6e87-d4af-4ee9-9b18-9e7dcc5c8eaf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:30:35.492951 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bba6e87-d4af-4ee9-9b18-9e7dcc5c8eaf admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:30:35.509487 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1368/2739] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:30:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:30:35.519905 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894fd285-1c84-4542-9bc4-d1a71b9799b6 None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:30:35.521621 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-894fd285-1c84-4542-9bc4-d1a71b9799b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:30:35.580488 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894fd285-1c84-4542-9bc4-d1a71b9799b6 None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:30:35.584486 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8307a6c-e8e4-480f-89de-cc9da6c3ea8d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0817c3320d5d41d8bba0550c8ae8dd60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:30:35.585517 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8307a6c-e8e4-480f-89de-cc9da6c3ea8d service neutron] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:35.585797 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8307a6c-e8e4-480f-89de-cc9da6c3ea8d service neutron] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:35.586073 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8307a6c-e8e4-480f-89de-cc9da6c3ea8d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:35.587592 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1372/2740] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 01:30:35 2022] POST /identity/v3/users => generated 297 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:30:35.591507 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bba6e87-d4af-4ee9-9b18-9e7dcc5c8eaf admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:30:35.595971 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8307a6c-e8e4-480f-89de-cc9da6c3ea8d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:30:35.597500 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8307a6c-e8e4-480f-89de-cc9da6c3ea8d service neutron] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:30:35.600989 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bba6e87-d4af-4ee9-9b18-9e7dcc5c8eaf admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:30:35.608686 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1369/2741] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:30:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 01:30:35.608933 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d56be20-1e7a-45bc-bc8b-f53b69adccf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:30:35.609367 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d56be20-1e7a-45bc-bc8b-f53b69adccf6 admin admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:35.609652 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d56be20-1e7a-45bc-bc8b-f53b69adccf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:35.609989 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d56be20-1e7a-45bc-bc8b-f53b69adccf6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:35.610640 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d56be20-1e7a-45bc-bc8b-f53b69adccf6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:30:35.612276 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d56be20-1e7a-45bc-bc8b-f53b69adccf6 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:30:35.625891 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1373/2742] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:30:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 01:30:35.627363 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8307a6c-e8e4-480f-89de-cc9da6c3ea8d service neutron] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:30:35.637270 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8307a6c-e8e4-480f-89de-cc9da6c3ea8d service neutron] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:30:35.644187 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d98db24-ad42-4947-ae40-b0fc45a38646 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:30:35.644668 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d98db24-ad42-4947-ae40-b0fc45a38646 admin admin] REQUEST_METHOD: `PUT` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:35.644987 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d98db24-ad42-4947-ae40-b0fc45a38646 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:35.645287 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d98db24-ad42-4947-ae40-b0fc45a38646 admin admin] PATH_INFO: `/v3/projects/4159edec54c040b2aeb67767180af781/users/120c456848a749afb9faa0e3f8dcb612/roles/d009411496b8452b88e77ffb8db015a7` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:35.661652 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d98db24-ad42-4947-ae40-b0fc45a38646 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4159edec54c040b2aeb67767180af781, user_id=120c456848a749afb9faa0e3f8dcb612, role_id=d009411496b8452b88e77ffb8db015a7)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:30:35.664238 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d98db24-ad42-4947-ae40-b0fc45a38646 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:30:35.753422 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1370/2743] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 01:30:35 2022] PUT /identity/v3/projects/4159edec54c040b2aeb67767180af781/users/120c456848a749afb9faa0e3f8dcb612/roles/d009411496b8452b88e77ffb8db015a7 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:30:35.757390 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d56be20-1e7a-45bc-bc8b-f53b69adccf6 admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:30:35.767362 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d56be20-1e7a-45bc-bc8b-f53b69adccf6 admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:30:35.827489 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40c9f64-bf9d-4d7b-87a8-4dc318a1d538 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:30:35.828402 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40c9f64-bf9d-4d7b-87a8-4dc318a1d538 admin admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:35.828929 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40c9f64-bf9d-4d7b-87a8-4dc318a1d538 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:35.829538 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40c9f64-bf9d-4d7b-87a8-4dc318a1d538 admin admin] PATH_INFO: `/v3/roles` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:35.830430 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40c9f64-bf9d-4d7b-87a8-4dc318a1d538 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:30:35.832171 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40c9f64-bf9d-4d7b-87a8-4dc318a1d538 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:30:35.843115 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1374/2744] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:30:35 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 01:30:35.848238 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d98db24-ad42-4947-ae40-b0fc45a38646 admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:30:35.860139 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d98db24-ad42-4947-ae40-b0fc45a38646 admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:30:35.864716 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfc9d99-82d5-47ec-8355-bd78f0d36241 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:30:35.865406 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfc9d99-82d5-47ec-8355-bd78f0d36241 admin admin] REQUEST_METHOD: `PUT` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:35.865862 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfc9d99-82d5-47ec-8355-bd78f0d36241 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:35.866139 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfc9d99-82d5-47ec-8355-bd78f0d36241 admin admin] PATH_INFO: `/v3/projects/4159edec54c040b2aeb67767180af781/users/120c456848a749afb9faa0e3f8dcb612/roles/d009411496b8452b88e77ffb8db015a7` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:35.868021 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfc9d99-82d5-47ec-8355-bd78f0d36241 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4159edec54c040b2aeb67767180af781, user_id=120c456848a749afb9faa0e3f8dcb612, role_id=d009411496b8452b88e77ffb8db015a7)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:30:35.870666 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfc9d99-82d5-47ec-8355-bd78f0d36241 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:30:35.881564 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1371/2745] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 01:30:35 2022] PUT /identity/v3/projects/4159edec54c040b2aeb67767180af781/users/120c456848a749afb9faa0e3f8dcb612/roles/d009411496b8452b88e77ffb8db015a7 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:30:35.899170 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac75e87-bcad-44ee-987f-3a4aad93aa5f None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:35.899692 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac75e87-bcad-44ee-987f-3a4aad93aa5f None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:35.900036 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac75e87-bcad-44ee-987f-3a4aad93aa5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:35.930962 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-8ac75e87-bcad-44ee-987f-3a4aad93aa5f None None] MFA Rules not processed for user `120c456848a749afb9faa0e3f8dcb612`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:30:35.932179 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-8ac75e87-bcad-44ee-987f-3a4aad93aa5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:30:36.005218 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1375/2746] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:30:35 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:30:36.016646 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfc9d99-82d5-47ec-8355-bd78f0d36241 admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:30:36.018353 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-7cfc9d99-82d5-47ec-8355-bd78f0d36241 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:30:36.064433 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac75e87-bcad-44ee-987f-3a4aad93aa5f None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:30:36.066011 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-8ac75e87-bcad-44ee-987f-3a4aad93aa5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:30:36.177161 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:30:36.188283 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac75e87-bcad-44ee-987f-3a4aad93aa5f None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:30:36.188629 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:30:36.189981 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ec2136-3d24-4f91-ae57-fe8399388435 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:30:36.190647 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ec2136-3d24-4f91-ae57-fe8399388435 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:36.190936 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ec2136-3d24-4f91-ae57-fe8399388435 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:36.191212 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ec2136-3d24-4f91-ae57-fe8399388435 admin admin] PATH_INFO: `/v3/projects/765aab3ab0ed4b09815eae61702a4155` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:36.192400 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ec2136-3d24-4f91-ae57-fe8399388435 admin admin] RBAC: Authorizing `identity:delete_project(project_id=765aab3ab0ed4b09815eae61702a4155)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:30:36.194770 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ec2136-3d24-4f91-ae57-fe8399388435 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:30:36.225790 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfc9d99-82d5-47ec-8355-bd78f0d36241 admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:30:36.229877 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c314608-8244-4e6f-acfa-70cfd2c7be04 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5c8fc69c518412ba1aad61a9f4f558c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:30:36.230494 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c314608-8244-4e6f-acfa-70cfd2c7be04 service cinder] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:36.230829 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c314608-8244-4e6f-acfa-70cfd2c7be04 service cinder] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:36.231106 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c314608-8244-4e6f-acfa-70cfd2c7be04 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:36.240589 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c314608-8244-4e6f-acfa-70cfd2c7be04 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:30:36.242138 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c314608-8244-4e6f-acfa-70cfd2c7be04 service cinder] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:30:36.252007 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1372/2747] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:30:36 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:30:36.345535 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-e3ec2136-3d24-4f91-ae57-fe8399388435 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '765aab3ab0ed4b09815eae61702a4155', 'request_id': 'req-e3ec2136-3d24-4f91-ae57-fe8399388435'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:36.351207 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-e3ec2136-3d24-4f91-ae57-fe8399388435 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '765aab3ab0ed4b09815eae61702a4155', 'request_id': 'req-e3ec2136-3d24-4f91-ae57-fe8399388435'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:36.362831 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c314608-8244-4e6f-acfa-70cfd2c7be04 service cinder] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:30:36.371782 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c314608-8244-4e6f-acfa-70cfd2c7be04 service cinder] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:30:36.421808 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4a46ca-b024-4ce5-9c3f-447162f16993 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5c8fc69c518412ba1aad61a9f4f558c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:30:36.422444 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4a46ca-b024-4ce5-9c3f-447162f16993 service cinder] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:36.422731 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4a46ca-b024-4ce5-9c3f-447162f16993 service cinder] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:36.423006 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4a46ca-b024-4ce5-9c3f-447162f16993 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:36.424740 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-da4a46ca-b024-4ce5-9c3f-447162f16993 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:30:36.426853 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-e3ec2136-3d24-4f91-ae57-fe8399388435 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '765aab3ab0ed4b09815eae61702a4155', 'request_id': 'req-e3ec2136-3d24-4f91-ae57-fe8399388435'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:36.442645 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-e3ec2136-3d24-4f91-ae57-fe8399388435 admin admin] The token cache is being invalidate because project 765aab3ab0ed4b09815eae61702a4155 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81750) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 01:30:36.442857 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-e3ec2136-3d24-4f91-ae57-fe8399388435 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:36.443527 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-e3ec2136-3d24-4f91-ae57-fe8399388435 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:36.445409 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1376/2748] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 01:30:36 2022] DELETE /identity/v3/projects/765aab3ab0ed4b09815eae61702a4155 => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:30:36.448912 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ec2136-3d24-4f91-ae57-fe8399388435 admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:30:36.450413 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-e3ec2136-3d24-4f91-ae57-fe8399388435 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:30:36.502976 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4a46ca-b024-4ce5-9c3f-447162f16993 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:30:36.505530 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4a46ca-b024-4ce5-9c3f-447162f16993 service cinder] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:30:36.518775 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1373/2749] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:30:36 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:30:36.530320 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ec2136-3d24-4f91-ae57-fe8399388435 admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:30:36.535592 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b460a10-6175-4b52-8ffe-daa9e973cf99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:30:36.536229 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b460a10-6175-4b52-8ffe-daa9e973cf99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:36.536534 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b460a10-6175-4b52-8ffe-daa9e973cf99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:36.536830 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b460a10-6175-4b52-8ffe-daa9e973cf99 admin admin] PATH_INFO: `/v3/users/0cfd18c7e5514c2f97e8b502005be024` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:36.538248 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b460a10-6175-4b52-8ffe-daa9e973cf99 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0cfd18c7e5514c2f97e8b502005be024)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:30:36.540086 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b460a10-6175-4b52-8ffe-daa9e973cf99 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:30:36.755376 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-7b460a10-6175-4b52-8ffe-daa9e973cf99 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0cfd18c7e5514c2f97e8b502005be024'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:36.767572 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-7b460a10-6175-4b52-8ffe-daa9e973cf99 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0cfd18c7e5514c2f97e8b502005be024'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:36.768588 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-7b460a10-6175-4b52-8ffe-daa9e973cf99 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0cfd18c7e5514c2f97e8b502005be024'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:36.775522 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-7b460a10-6175-4b52-8ffe-daa9e973cf99 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0cfd18c7e5514c2f97e8b502005be024'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:36.849235 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-7b460a10-6175-4b52-8ffe-daa9e973cf99 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0cfd18c7e5514c2f97e8b502005be024'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:36.855466 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1377/2750] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 01:30:36 2022] DELETE /identity/v3/users/0cfd18c7e5514c2f97e8b502005be024 => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 01:30:37.443161 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4a46ca-b024-4ce5-9c3f-447162f16993 service cinder] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:30:37.463845 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4a46ca-b024-4ce5-9c3f-447162f16993 service cinder] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:30:37.529375 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b460a10-6175-4b52-8ffe-daa9e973cf99 admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:30:37.531631 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-7b460a10-6175-4b52-8ffe-daa9e973cf99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:30:37.543394 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39406e7a-fe45-4219-8334-10c4ba6ce3cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:30:37.544542 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39406e7a-fe45-4219-8334-10c4ba6ce3cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:37.544995 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39406e7a-fe45-4219-8334-10c4ba6ce3cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:37.545404 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39406e7a-fe45-4219-8334-10c4ba6ce3cd admin admin] PATH_INFO: `/v3/projects/38d4e29dd8204139b6f76a48d18ec82e` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:37.546768 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39406e7a-fe45-4219-8334-10c4ba6ce3cd admin admin] RBAC: Authorizing `identity:delete_project(project_id=38d4e29dd8204139b6f76a48d18ec82e)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:30:37.548938 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39406e7a-fe45-4219-8334-10c4ba6ce3cd admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:30:37.549867 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:30:37.573346 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:30:37.632386 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b460a10-6175-4b52-8ffe-daa9e973cf99 admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:30:37.637252 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3344b690-69b1-45e5-b798-c96190c258f7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5c8fc69c518412ba1aad61a9f4f558c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:30:37.638085 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3344b690-69b1-45e5-b798-c96190c258f7 service cinder] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:37.638344 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3344b690-69b1-45e5-b798-c96190c258f7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:37.638616 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3344b690-69b1-45e5-b798-c96190c258f7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:37.640459 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-3344b690-69b1-45e5-b798-c96190c258f7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:30:37.685830 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-39406e7a-fe45-4219-8334-10c4ba6ce3cd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '38d4e29dd8204139b6f76a48d18ec82e', 'request_id': 'req-39406e7a-fe45-4219-8334-10c4ba6ce3cd'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:37.691668 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-39406e7a-fe45-4219-8334-10c4ba6ce3cd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '38d4e29dd8204139b6f76a48d18ec82e', 'request_id': 'req-39406e7a-fe45-4219-8334-10c4ba6ce3cd'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:37.700708 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-39406e7a-fe45-4219-8334-10c4ba6ce3cd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '38d4e29dd8204139b6f76a48d18ec82e', 'request_id': 'req-39406e7a-fe45-4219-8334-10c4ba6ce3cd'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:37.721850 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3344b690-69b1-45e5-b798-c96190c258f7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:30:37.723025 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3344b690-69b1-45e5-b798-c96190c258f7 service cinder] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:30:37.735254 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1378/2751] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:30:37 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 01:30:37.809192 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-39406e7a-fe45-4219-8334-10c4ba6ce3cd admin admin] The token cache is being invalidate because project 38d4e29dd8204139b6f76a48d18ec82e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81749) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 01:30:37.809495 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-39406e7a-fe45-4219-8334-10c4ba6ce3cd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:37.810181 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-39406e7a-fe45-4219-8334-10c4ba6ce3cd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:37.814891 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1374/2752] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 01:30:37 2022] DELETE /identity/v3/projects/38d4e29dd8204139b6f76a48d18ec82e => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:30:37.816305 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3344b690-69b1-45e5-b798-c96190c258f7 service cinder] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:30:37.817871 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-3344b690-69b1-45e5-b798-c96190c258f7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:30:37.833644 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3344b690-69b1-45e5-b798-c96190c258f7 service cinder] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:30:37.890577 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259f3eb9-6280-4f18-8b11-a608c7fbf194 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:30:37.891224 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259f3eb9-6280-4f18-8b11-a608c7fbf194 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:37.891504 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259f3eb9-6280-4f18-8b11-a608c7fbf194 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:37.891777 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259f3eb9-6280-4f18-8b11-a608c7fbf194 admin admin] PATH_INFO: `/v3/users/6923c98eec7b4c47a21d40c7f471d706` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:37.892976 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259f3eb9-6280-4f18-8b11-a608c7fbf194 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6923c98eec7b4c47a21d40c7f471d706)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:30:37.894968 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259f3eb9-6280-4f18-8b11-a608c7fbf194 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:30:38.121176 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-259f3eb9-6280-4f18-8b11-a608c7fbf194 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6923c98eec7b4c47a21d40c7f471d706'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:38.132574 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-259f3eb9-6280-4f18-8b11-a608c7fbf194 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6923c98eec7b4c47a21d40c7f471d706'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:38.133299 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-259f3eb9-6280-4f18-8b11-a608c7fbf194 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6923c98eec7b4c47a21d40c7f471d706'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:38.140768 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-259f3eb9-6280-4f18-8b11-a608c7fbf194 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6923c98eec7b4c47a21d40c7f471d706'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:38.232167 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-259f3eb9-6280-4f18-8b11-a608c7fbf194 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6923c98eec7b4c47a21d40c7f471d706'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:38.238611 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1379/2753] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 01:30:37 2022] DELETE /identity/v3/users/6923c98eec7b4c47a21d40c7f471d706 => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:30:38.438769 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:30:38.777829 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39406e7a-fe45-4219-8334-10c4ba6ce3cd admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:30:38.779937 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-39406e7a-fe45-4219-8334-10c4ba6ce3cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:30:38.801087 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39406e7a-fe45-4219-8334-10c4ba6ce3cd admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:30:38.881581 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d093592c-2629-4811-9a54-0322fc4d900b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:30:38.882424 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d093592c-2629-4811-9a54-0322fc4d900b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:38.882783 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d093592c-2629-4811-9a54-0322fc4d900b admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:38.883133 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d093592c-2629-4811-9a54-0322fc4d900b admin admin] PATH_INFO: `/v3/projects/749026c7ea74462f8bed6cd0b4843fa6` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:38.884337 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d093592c-2629-4811-9a54-0322fc4d900b admin admin] RBAC: Authorizing `identity:delete_project(project_id=749026c7ea74462f8bed6cd0b4843fa6)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:30:38.886344 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d093592c-2629-4811-9a54-0322fc4d900b admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:30:38.998555 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-d093592c-2629-4811-9a54-0322fc4d900b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '749026c7ea74462f8bed6cd0b4843fa6', 'request_id': 'req-d093592c-2629-4811-9a54-0322fc4d900b'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:39.006831 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-d093592c-2629-4811-9a54-0322fc4d900b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '749026c7ea74462f8bed6cd0b4843fa6', 'request_id': 'req-d093592c-2629-4811-9a54-0322fc4d900b'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:39.015341 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-d093592c-2629-4811-9a54-0322fc4d900b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '749026c7ea74462f8bed6cd0b4843fa6', 'request_id': 'req-d093592c-2629-4811-9a54-0322fc4d900b'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:39.111660 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-d093592c-2629-4811-9a54-0322fc4d900b admin admin] The token cache is being invalidate because project 749026c7ea74462f8bed6cd0b4843fa6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81749) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 01:30:39.112398 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-d093592c-2629-4811-9a54-0322fc4d900b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:39.114074 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-d093592c-2629-4811-9a54-0322fc4d900b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:39.121948 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee2f8b7-e18c-4c06-8b76-bd3ec919bc12 None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:39.122966 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee2f8b7-e18c-4c06-8b76-bd3ec919bc12 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:39.123923 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee2f8b7-e18c-4c06-8b76-bd3ec919bc12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:39.126783 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1375/2754] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 01:30:38 2022] DELETE /identity/v3/projects/749026c7ea74462f8bed6cd0b4843fa6 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:30:39.155368 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-5ee2f8b7-e18c-4c06-8b76-bd3ec919bc12 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:30:39.157444 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-5ee2f8b7-e18c-4c06-8b76-bd3ec919bc12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:30:39.256209 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1380/2755] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:30:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:30:39.263140 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7af0bb-4009-415f-b5f9-2002488b03cc None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:39.263635 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7af0bb-4009-415f-b5f9-2002488b03cc None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:39.264779 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7af0bb-4009-415f-b5f9-2002488b03cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:39.306065 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-fa7af0bb-4009-415f-b5f9-2002488b03cc None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:30:39.307800 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-fa7af0bb-4009-415f-b5f9-2002488b03cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:30:39.319798 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1376/2756] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:30:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 01:30:39.324262 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee2f8b7-e18c-4c06-8b76-bd3ec919bc12 None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:30:39.343941 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee2f8b7-e18c-4c06-8b76-bd3ec919bc12 None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:30:39.350164 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ffc31a-7a89-429b-88d4-891adcc940b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:30:39.350880 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ffc31a-7a89-429b-88d4-891adcc940b3 admin admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:39.351168 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ffc31a-7a89-429b-88d4-891adcc940b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:39.351441 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ffc31a-7a89-429b-88d4-891adcc940b3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:39.352235 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ffc31a-7a89-429b-88d4-891adcc940b3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:30:39.352548 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ffc31a-7a89-429b-88d4-891adcc940b3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:30:39.356041 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ffc31a-7a89-429b-88d4-891adcc940b3 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:30:39.376504 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1381/2757] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Aug 31 01:30:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 01:30:39.380041 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7af0bb-4009-415f-b5f9-2002488b03cc None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:30:39.391102 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7af0bb-4009-415f-b5f9-2002488b03cc None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:30:39.396537 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67aa07df-6b60-4313-b936-2d9f8e16b624 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:30:39.397355 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67aa07df-6b60-4313-b936-2d9f8e16b624 admin admin] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:39.397779 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67aa07df-6b60-4313-b936-2d9f8e16b624 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:39.398249 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67aa07df-6b60-4313-b936-2d9f8e16b624 admin admin] PATH_INFO: `/v3/projects` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:39.399178 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67aa07df-6b60-4313-b936-2d9f8e16b624 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:30:39.400084 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ffc31a-7a89-429b-88d4-891adcc940b3 admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:30:39.401583 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67aa07df-6b60-4313-b936-2d9f8e16b624 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:30:39.402098 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-26ffc31a-7a89-429b-88d4-891adcc940b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:30:39.464903 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ffc31a-7a89-429b-88d4-891adcc940b3 admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:30:39.468760 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091df943-a2c4-415f-95c7-a5c453968cfb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5c8fc69c518412ba1aad61a9f4f558c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:30:39.469402 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091df943-a2c4-415f-95c7-a5c453968cfb service cinder] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:39.469684 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091df943-a2c4-415f-95c7-a5c453968cfb service cinder] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:39.470001 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091df943-a2c4-415f-95c7-a5c453968cfb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:39.472021 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-091df943-a2c4-415f-95c7-a5c453968cfb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:30:39.486676 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1377/2758] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 01:30:39 2022] POST /identity/v3/projects => generated 375 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:30:39.491823 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67aa07df-6b60-4313-b936-2d9f8e16b624 admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:30:39.497656 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:30:39.577947 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091df943-a2c4-415f-95c7-a5c453968cfb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:30:39.581006 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091df943-a2c4-415f-95c7-a5c453968cfb service cinder] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:30:39.593266 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67aa07df-6b60-4313-b936-2d9f8e16b624 admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:30:39.602828 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da55c61-01c9-4667-9f27-94239d488f9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:30:39.603886 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da55c61-01c9-4667-9f27-94239d488f9e admin admin] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:39.603886 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da55c61-01c9-4667-9f27-94239d488f9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:39.604128 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da55c61-01c9-4667-9f27-94239d488f9e admin admin] PATH_INFO: `/v3/users` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:39.604707 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da55c61-01c9-4667-9f27-94239d488f9e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:30:39.605074 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1382/2759] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:30:39 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 01:30:39.606743 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da55c61-01c9-4667-9f27-94239d488f9e admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:30:39.695096 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1378/2760] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 01:30:39 2022] POST /identity/v3/users => generated 288 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:30:39.698698 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091df943-a2c4-415f-95c7-a5c453968cfb service cinder] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:30:39.713492 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091df943-a2c4-415f-95c7-a5c453968cfb service cinder] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:30:39.721190 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3de7631-ddc5-4441-a629-b77d5b6b50e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:30:39.721822 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3de7631-ddc5-4441-a629-b77d5b6b50e9 admin admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:39.722127 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3de7631-ddc5-4441-a629-b77d5b6b50e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:39.722378 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3de7631-ddc5-4441-a629-b77d5b6b50e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:39.724669 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3de7631-ddc5-4441-a629-b77d5b6b50e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:30:39.733876 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3de7631-ddc5-4441-a629-b77d5b6b50e9 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:30:39.752078 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1383/2761] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:30:39 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:30:39.756920 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da55c61-01c9-4667-9f27-94239d488f9e admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:30:39.768138 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da55c61-01c9-4667-9f27-94239d488f9e admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:30:39.773603 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037507fd-300f-498e-ad63-4bce505bf3a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:30:39.774240 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037507fd-300f-498e-ad63-4bce505bf3a4 admin admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:39.774543 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037507fd-300f-498e-ad63-4bce505bf3a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:39.774817 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037507fd-300f-498e-ad63-4bce505bf3a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:39.775488 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037507fd-300f-498e-ad63-4bce505bf3a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:30:39.777141 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037507fd-300f-498e-ad63-4bce505bf3a4 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:30:39.791631 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1379/2762] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:30:39 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:30:39.794235 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3de7631-ddc5-4441-a629-b77d5b6b50e9 admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:30:39.812909 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3de7631-ddc5-4441-a629-b77d5b6b50e9 admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:30:39.818765 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71972dec-6430-49e9-b7dc-a36ab803d6a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:30:39.824266 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71972dec-6430-49e9-b7dc-a36ab803d6a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:39.824266 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71972dec-6430-49e9-b7dc-a36ab803d6a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:39.824266 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71972dec-6430-49e9-b7dc-a36ab803d6a8 admin admin] PATH_INFO: `/v3/projects/ee99d9b8ed0f4e9297aff9e64fa53ff7/users/9142f96b5780486389e271ae19423349/roles/d0b9768f4d014766b9326387f2e11f34` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:39.824266 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:30:39.841677 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71972dec-6430-49e9-b7dc-a36ab803d6a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee99d9b8ed0f4e9297aff9e64fa53ff7, user_id=9142f96b5780486389e271ae19423349, role_id=d0b9768f4d014766b9326387f2e11f34)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:30:39.845017 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71972dec-6430-49e9-b7dc-a36ab803d6a8 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:30:39.932176 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037507fd-300f-498e-ad63-4bce505bf3a4 admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:30:39.932176 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-037507fd-300f-498e-ad63-4bce505bf3a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:30:39.932176 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037507fd-300f-498e-ad63-4bce505bf3a4 admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:30:39.935161 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b665ea5c-5801-42e8-a0f3-0ff5e0b33037 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:30:39.935846 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b665ea5c-5801-42e8-a0f3-0ff5e0b33037 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:39.936131 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b665ea5c-5801-42e8-a0f3-0ff5e0b33037 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:39.936409 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b665ea5c-5801-42e8-a0f3-0ff5e0b33037 admin admin] PATH_INFO: `/v3/users/1caf2edea599429c96cdc90b240bcb78` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:39.937703 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b665ea5c-5801-42e8-a0f3-0ff5e0b33037 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1caf2edea599429c96cdc90b240bcb78)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:30:39.941262 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b665ea5c-5801-42e8-a0f3-0ff5e0b33037 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:30:39.996808 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1384/2763] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 01:30:39 2022] PUT /identity/v3/projects/ee99d9b8ed0f4e9297aff9e64fa53ff7/users/9142f96b5780486389e271ae19423349/roles/d0b9768f4d014766b9326387f2e11f34 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 01:30:40.000685 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20aac363-d0eb-4c34-bfdc-78f282af57bc None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:40.001033 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20aac363-d0eb-4c34-bfdc-78f282af57bc None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:40.001357 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20aac363-d0eb-4c34-bfdc-78f282af57bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:40.030382 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-20aac363-d0eb-4c34-bfdc-78f282af57bc None None] MFA Rules not processed for user `9142f96b5780486389e271ae19423349`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:30:40.031554 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-20aac363-d0eb-4c34-bfdc-78f282af57bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:30:40.109429 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1385/2764] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:30:39 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 01:30:40.110835 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-b665ea5c-5801-42e8-a0f3-0ff5e0b33037 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1caf2edea599429c96cdc90b240bcb78'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:40.121932 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20aac363-d0eb-4c34-bfdc-78f282af57bc None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:30:40.185888 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20aac363-d0eb-4c34-bfdc-78f282af57bc None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:30:40.189261 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-b665ea5c-5801-42e8-a0f3-0ff5e0b33037 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1caf2edea599429c96cdc90b240bcb78'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:40.193068 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c558d481-9e8b-4f81-bdbd-f17901dd85fa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5c8fc69c518412ba1aad61a9f4f558c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:30:40.193606 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c558d481-9e8b-4f81-bdbd-f17901dd85fa service cinder] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:40.193885 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c558d481-9e8b-4f81-bdbd-f17901dd85fa service cinder] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:40.194389 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c558d481-9e8b-4f81-bdbd-f17901dd85fa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:40.196065 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-b665ea5c-5801-42e8-a0f3-0ff5e0b33037 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1caf2edea599429c96cdc90b240bcb78'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:40.207750 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c558d481-9e8b-4f81-bdbd-f17901dd85fa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:30:40.210148 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c558d481-9e8b-4f81-bdbd-f17901dd85fa service cinder] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:30:40.213879 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-b665ea5c-5801-42e8-a0f3-0ff5e0b33037 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1caf2edea599429c96cdc90b240bcb78'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:40.216937 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-b665ea5c-5801-42e8-a0f3-0ff5e0b33037 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1caf2edea599429c96cdc90b240bcb78'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:40.222613 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1386/2765] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:30:40 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:30:40.252606 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1380/2766] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 01:30:39 2022] DELETE /identity/v3/users/1caf2edea599429c96cdc90b240bcb78 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:30:40.257348 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa87ce70-4a4d-4670-86ae-71e94e152c26 None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:40.257854 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa87ce70-4a4d-4670-86ae-71e94e152c26 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:40.258132 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa87ce70-4a4d-4670-86ae-71e94e152c26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:40.282525 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-aa87ce70-4a4d-4670-86ae-71e94e152c26 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:30:40.285253 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-aa87ce70-4a4d-4670-86ae-71e94e152c26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:30:40.370080 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1387/2767] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:30:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:30:40.383373 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b665ea5c-5801-42e8-a0f3-0ff5e0b33037 admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:30:40.387591 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-b665ea5c-5801-42e8-a0f3-0ff5e0b33037 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:30:40.466750 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b665ea5c-5801-42e8-a0f3-0ff5e0b33037 admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:30:40.472017 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc15d2b2-ec94-4222-ac8b-c24b653025dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0817c3320d5d41d8bba0550c8ae8dd60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:30:40.473174 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc15d2b2-ec94-4222-ac8b-c24b653025dc service neutron] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:40.473467 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc15d2b2-ec94-4222-ac8b-c24b653025dc service neutron] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:40.473752 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc15d2b2-ec94-4222-ac8b-c24b653025dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:40.483781 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc15d2b2-ec94-4222-ac8b-c24b653025dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:30:40.485259 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc15d2b2-ec94-4222-ac8b-c24b653025dc service neutron] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:30:40.489221 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:30:40.495730 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1381/2768] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:30:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 01:30:40.908645 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa87ce70-4a4d-4670-86ae-71e94e152c26 None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:30:40.910389 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-aa87ce70-4a4d-4670-86ae-71e94e152c26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:30:40.929451 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa87ce70-4a4d-4670-86ae-71e94e152c26 None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:30:40.934799 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e775884e-ffea-46fd-9afa-5ff44acac94e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:30:40.937074 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e775884e-ffea-46fd-9afa-5ff44acac94e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:40.937494 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e775884e-ffea-46fd-9afa-5ff44acac94e admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:40.937956 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e775884e-ffea-46fd-9afa-5ff44acac94e admin admin] PATH_INFO: `/v3/projects/8c7016c9f3424dd385816e4d25b0d02d` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:40.939969 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e775884e-ffea-46fd-9afa-5ff44acac94e admin admin] RBAC: Authorizing `identity:delete_project(project_id=8c7016c9f3424dd385816e4d25b0d02d)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:30:40.946307 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e775884e-ffea-46fd-9afa-5ff44acac94e admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:30:40.960243 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:30:41.081521 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-e775884e-ffea-46fd-9afa-5ff44acac94e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8c7016c9f3424dd385816e4d25b0d02d', 'request_id': 'req-e775884e-ffea-46fd-9afa-5ff44acac94e'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:41.091133 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-e775884e-ffea-46fd-9afa-5ff44acac94e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8c7016c9f3424dd385816e4d25b0d02d', 'request_id': 'req-e775884e-ffea-46fd-9afa-5ff44acac94e'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:41.161729 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-e775884e-ffea-46fd-9afa-5ff44acac94e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8c7016c9f3424dd385816e4d25b0d02d', 'request_id': 'req-e775884e-ffea-46fd-9afa-5ff44acac94e'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:41.181541 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-e775884e-ffea-46fd-9afa-5ff44acac94e admin admin] The token cache is being invalidate because project 8c7016c9f3424dd385816e4d25b0d02d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81750) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 01:30:41.181848 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-e775884e-ffea-46fd-9afa-5ff44acac94e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:41.188299 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-e775884e-ffea-46fd-9afa-5ff44acac94e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:41.188299 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1388/2769] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 01:30:40 2022] DELETE /identity/v3/projects/8c7016c9f3424dd385816e4d25b0d02d => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:30:41.189261 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc15d2b2-ec94-4222-ac8b-c24b653025dc service neutron] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:30:41.206796 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc15d2b2-ec94-4222-ac8b-c24b653025dc service neutron] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:30:41.267930 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330a9559-b90d-4f4f-b72d-f527d256ff4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:30:41.268576 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330a9559-b90d-4f4f-b72d-f527d256ff4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:41.268896 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330a9559-b90d-4f4f-b72d-f527d256ff4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:41.269229 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330a9559-b90d-4f4f-b72d-f527d256ff4f admin admin] PATH_INFO: `/v3/users/120c456848a749afb9faa0e3f8dcb612` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:41.270400 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330a9559-b90d-4f4f-b72d-f527d256ff4f admin admin] RBAC: Authorizing `identity:delete_user(user_id=120c456848a749afb9faa0e3f8dcb612)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:30:41.272874 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330a9559-b90d-4f4f-b72d-f527d256ff4f admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:30:41.470670 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-330a9559-b90d-4f4f-b72d-f527d256ff4f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '120c456848a749afb9faa0e3f8dcb612'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:41.535008 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-330a9559-b90d-4f4f-b72d-f527d256ff4f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '120c456848a749afb9faa0e3f8dcb612'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:41.535585 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-330a9559-b90d-4f4f-b72d-f527d256ff4f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '120c456848a749afb9faa0e3f8dcb612'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:41.543955 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-330a9559-b90d-4f4f-b72d-f527d256ff4f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '120c456848a749afb9faa0e3f8dcb612'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:41.548150 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-330a9559-b90d-4f4f-b72d-f527d256ff4f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '120c456848a749afb9faa0e3f8dcb612'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:41.572415 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1382/2770] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 01:30:41 2022] DELETE /identity/v3/users/120c456848a749afb9faa0e3f8dcb612 => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:30:42.051711 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e775884e-ffea-46fd-9afa-5ff44acac94e admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:30:42.053206 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-e775884e-ffea-46fd-9afa-5ff44acac94e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:30:42.071167 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e775884e-ffea-46fd-9afa-5ff44acac94e admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:30:42.150148 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd111a7-30ea-4d96-9e32-0fd0cbec75d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:30:42.151789 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd111a7-30ea-4d96-9e32-0fd0cbec75d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:42.152275 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd111a7-30ea-4d96-9e32-0fd0cbec75d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:42.152907 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd111a7-30ea-4d96-9e32-0fd0cbec75d4 admin admin] PATH_INFO: `/v3/projects/4159edec54c040b2aeb67767180af781` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:42.154027 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd111a7-30ea-4d96-9e32-0fd0cbec75d4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4159edec54c040b2aeb67767180af781)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:30:42.156078 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd111a7-30ea-4d96-9e32-0fd0cbec75d4 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:30:42.339975 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-6dd111a7-30ea-4d96-9e32-0fd0cbec75d4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4159edec54c040b2aeb67767180af781', 'request_id': 'req-6dd111a7-30ea-4d96-9e32-0fd0cbec75d4'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:42.349210 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-6dd111a7-30ea-4d96-9e32-0fd0cbec75d4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4159edec54c040b2aeb67767180af781', 'request_id': 'req-6dd111a7-30ea-4d96-9e32-0fd0cbec75d4'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:42.426551 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-6dd111a7-30ea-4d96-9e32-0fd0cbec75d4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4159edec54c040b2aeb67767180af781', 'request_id': 'req-6dd111a7-30ea-4d96-9e32-0fd0cbec75d4'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:42.439119 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-6dd111a7-30ea-4d96-9e32-0fd0cbec75d4 admin admin] The token cache is being invalidate because project 4159edec54c040b2aeb67767180af781 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81750) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 01:30:42.439428 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-6dd111a7-30ea-4d96-9e32-0fd0cbec75d4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:42.440069 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-6dd111a7-30ea-4d96-9e32-0fd0cbec75d4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:42.441638 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1389/2771] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 01:30:42 2022] DELETE /identity/v3/projects/4159edec54c040b2aeb67767180af781 => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:30:42.446754 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64665324-1ac2-4601-a639-e36014087d3b None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:42.447285 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64665324-1ac2-4601-a639-e36014087d3b None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:42.447285 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64665324-1ac2-4601-a639-e36014087d3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:42.470203 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-64665324-1ac2-4601-a639-e36014087d3b None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:30:42.471692 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-64665324-1ac2-4601-a639-e36014087d3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:30:42.533565 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1383/2772] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:30:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 01:30:42.541365 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d1426c-3b85-4878-990c-9aa89786feb6 None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:42.541657 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d1426c-3b85-4878-990c-9aa89786feb6 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:42.541934 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d1426c-3b85-4878-990c-9aa89786feb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:42.561835 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-91d1426c-3b85-4878-990c-9aa89786feb6 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:30:42.563095 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-91d1426c-3b85-4878-990c-9aa89786feb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:30:42.576054 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1390/2773] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:30:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:30:42.581238 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64665324-1ac2-4601-a639-e36014087d3b None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:30:42.593837 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64665324-1ac2-4601-a639-e36014087d3b None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:30:42.598798 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4335a31f-7c06-44c4-baa3-4c128146ce30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:30:42.599411 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4335a31f-7c06-44c4-baa3-4c128146ce30 admin admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:42.599697 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4335a31f-7c06-44c4-baa3-4c128146ce30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:42.599978 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4335a31f-7c06-44c4-baa3-4c128146ce30 admin admin] PATH_INFO: `/v3/domains` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:42.600626 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4335a31f-7c06-44c4-baa3-4c128146ce30 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:30:42.600991 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4335a31f-7c06-44c4-baa3-4c128146ce30 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:30:42.602658 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4335a31f-7c06-44c4-baa3-4c128146ce30 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:30:42.615084 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1384/2774] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Aug 31 01:30:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:30:42.619554 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d1426c-3b85-4878-990c-9aa89786feb6 None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:30:42.629219 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d1426c-3b85-4878-990c-9aa89786feb6 None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:30:42.637769 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21334da-c989-4740-a7fe-c0acb2760238 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:30:42.637769 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21334da-c989-4740-a7fe-c0acb2760238 admin admin] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:42.637769 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21334da-c989-4740-a7fe-c0acb2760238 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:42.637769 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21334da-c989-4740-a7fe-c0acb2760238 admin admin] PATH_INFO: `/v3/projects` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:42.638251 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21334da-c989-4740-a7fe-c0acb2760238 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:30:42.638351 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21334da-c989-4740-a7fe-c0acb2760238 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:30:42.754381 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:30:42.761366 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1391/2775] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 01:30:42 2022] POST /identity/v3/projects => generated 355 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:30:42.766401 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4335a31f-7c06-44c4-baa3-4c128146ce30 admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:30:42.774529 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:30:42.780827 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4335a31f-7c06-44c4-baa3-4c128146ce30 admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:30:42.863656 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350b5b7a-0ebc-4722-8c17-0670b4e942d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:30:42.863656 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350b5b7a-0ebc-4722-8c17-0670b4e942d9 admin admin] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:42.863656 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350b5b7a-0ebc-4722-8c17-0670b4e942d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:42.863656 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350b5b7a-0ebc-4722-8c17-0670b4e942d9 admin admin] PATH_INFO: `/v3/users` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:42.864046 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350b5b7a-0ebc-4722-8c17-0670b4e942d9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:30:42.864046 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350b5b7a-0ebc-4722-8c17-0670b4e942d9 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:30:42.943605 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1385/2776] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 01:30:42 2022] POST /identity/v3/users => generated 278 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:30:42.947695 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21334da-c989-4740-a7fe-c0acb2760238 admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:30:42.963029 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21334da-c989-4740-a7fe-c0acb2760238 admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:30:42.974848 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673febc7-7c76-4cce-aefa-df979916dc42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:30:42.981645 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673febc7-7c76-4cce-aefa-df979916dc42 admin admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:42.981645 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673febc7-7c76-4cce-aefa-df979916dc42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:42.981645 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673febc7-7c76-4cce-aefa-df979916dc42 admin admin] PATH_INFO: `/v3/roles` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:42.981645 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673febc7-7c76-4cce-aefa-df979916dc42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:30:42.981645 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673febc7-7c76-4cce-aefa-df979916dc42 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:30:42.996005 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1392/2777] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:30:42 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:30:43.000972 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350b5b7a-0ebc-4722-8c17-0670b4e942d9 admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:30:43.011020 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350b5b7a-0ebc-4722-8c17-0670b4e942d9 admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:30:43.016636 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826ca970-a248-477e-9a1e-0f87e0113856 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:30:43.017346 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826ca970-a248-477e-9a1e-0f87e0113856 admin admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:43.017661 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826ca970-a248-477e-9a1e-0f87e0113856 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:43.017983 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826ca970-a248-477e-9a1e-0f87e0113856 admin admin] PATH_INFO: `/v3/roles` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:43.018610 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826ca970-a248-477e-9a1e-0f87e0113856 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:30:43.020367 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826ca970-a248-477e-9a1e-0f87e0113856 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:30:43.036015 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1386/2778] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:30:42 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:30:43.038298 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673febc7-7c76-4cce-aefa-df979916dc42 admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:30:43.058923 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673febc7-7c76-4cce-aefa-df979916dc42 admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:30:43.060003 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea280af4-045d-45d8-83fa-5524a07ed817 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:30:43.061327 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea280af4-045d-45d8-83fa-5524a07ed817 admin admin] REQUEST_METHOD: `PUT` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:43.061555 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea280af4-045d-45d8-83fa-5524a07ed817 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:43.061836 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea280af4-045d-45d8-83fa-5524a07ed817 admin admin] PATH_INFO: `/v3/projects/600c8e990f9347debe7491f19214f555/users/fa7f3de1f1974cb8b527bfa3b9e3a1cc/roles/d0b9768f4d014766b9326387f2e11f34` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:43.082801 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea280af4-045d-45d8-83fa-5524a07ed817 admin admin] RBAC: Authorizing `identity:create_grant(project_id=600c8e990f9347debe7491f19214f555, user_id=fa7f3de1f1974cb8b527bfa3b9e3a1cc, role_id=d0b9768f4d014766b9326387f2e11f34)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:30:43.088180 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea280af4-045d-45d8-83fa-5524a07ed817 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:30:43.178090 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1393/2779] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 01:30:43 2022] PUT /identity/v3/projects/600c8e990f9347debe7491f19214f555/users/fa7f3de1f1974cb8b527bfa3b9e3a1cc/roles/d0b9768f4d014766b9326387f2e11f34 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:30:43.189321 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580bd2a7-f455-48db-91a1-70223054e6cc None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:43.189659 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580bd2a7-f455-48db-91a1-70223054e6cc None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:43.189945 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580bd2a7-f455-48db-91a1-70223054e6cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:43.219044 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-580bd2a7-f455-48db-91a1-70223054e6cc None None] MFA Rules not processed for user `fa7f3de1f1974cb8b527bfa3b9e3a1cc`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:30:43.220247 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-580bd2a7-f455-48db-91a1-70223054e6cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:30:43.298630 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1387/2780] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:30:43 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:30:43.340242 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea280af4-045d-45d8-83fa-5524a07ed817 admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:30:43.342272 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-ea280af4-045d-45d8-83fa-5524a07ed817 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:30:43.441111 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea280af4-045d-45d8-83fa-5524a07ed817 admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:30:43.445278 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8eacb0-ad23-4c4d-b8e3-8ccd74d2254e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5c8fc69c518412ba1aad61a9f4f558c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:30:43.446089 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8eacb0-ad23-4c4d-b8e3-8ccd74d2254e service cinder] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:43.446376 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8eacb0-ad23-4c4d-b8e3-8ccd74d2254e service cinder] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:43.446657 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8eacb0-ad23-4c4d-b8e3-8ccd74d2254e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:43.456019 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8eacb0-ad23-4c4d-b8e3-8ccd74d2254e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:30:43.457664 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8eacb0-ad23-4c4d-b8e3-8ccd74d2254e service cinder] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:30:43.467620 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1394/2781] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:30:43 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:30:43.486132 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580bd2a7-f455-48db-91a1-70223054e6cc None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:30:43.553844 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580bd2a7-f455-48db-91a1-70223054e6cc None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:30:43.558625 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77aa1443-bd1b-49a3-b635-8aac6118d7d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:30:43.559269 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77aa1443-bd1b-49a3-b635-8aac6118d7d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:43.559552 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77aa1443-bd1b-49a3-b635-8aac6118d7d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:43.559825 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77aa1443-bd1b-49a3-b635-8aac6118d7d4 admin admin] PATH_INFO: `/v3/users/fa7f3de1f1974cb8b527bfa3b9e3a1cc` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:43.561046 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77aa1443-bd1b-49a3-b635-8aac6118d7d4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fa7f3de1f1974cb8b527bfa3b9e3a1cc)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:30:43.563025 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77aa1443-bd1b-49a3-b635-8aac6118d7d4 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:30:43.577463 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:30:43.904409 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-77aa1443-bd1b-49a3-b635-8aac6118d7d4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fa7f3de1f1974cb8b527bfa3b9e3a1cc'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:43.969834 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-77aa1443-bd1b-49a3-b635-8aac6118d7d4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fa7f3de1f1974cb8b527bfa3b9e3a1cc'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:43.970340 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-77aa1443-bd1b-49a3-b635-8aac6118d7d4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fa7f3de1f1974cb8b527bfa3b9e3a1cc'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:43.979760 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-77aa1443-bd1b-49a3-b635-8aac6118d7d4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fa7f3de1f1974cb8b527bfa3b9e3a1cc'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:43.982596 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-77aa1443-bd1b-49a3-b635-8aac6118d7d4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fa7f3de1f1974cb8b527bfa3b9e3a1cc'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:44.012280 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1388/2782] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 01:30:43 2022] DELETE /identity/v3/users/fa7f3de1f1974cb8b527bfa3b9e3a1cc => generated 0 bytes in 527 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:30:44.017365 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ff9d64-c78e-4646-b183-87f3f6284dd7 None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:44.017655 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ff9d64-c78e-4646-b183-87f3f6284dd7 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:44.017951 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ff9d64-c78e-4646-b183-87f3f6284dd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:44.050394 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-b8ff9d64-c78e-4646-b183-87f3f6284dd7 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:30:44.052153 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-b8ff9d64-c78e-4646-b183-87f3f6284dd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:30:44.137433 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1395/2783] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:30:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:30:44.147932 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77aa1443-bd1b-49a3-b635-8aac6118d7d4 admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:30:44.149629 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-77aa1443-bd1b-49a3-b635-8aac6118d7d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:30:44.213797 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77aa1443-bd1b-49a3-b635-8aac6118d7d4 admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:30:44.219135 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066cc662-7f4b-4d44-86dc-3d035523ebc6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0817c3320d5d41d8bba0550c8ae8dd60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:30:44.219769 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066cc662-7f4b-4d44-86dc-3d035523ebc6 service neutron] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:44.220049 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066cc662-7f4b-4d44-86dc-3d035523ebc6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:44.220335 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066cc662-7f4b-4d44-86dc-3d035523ebc6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:44.233828 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066cc662-7f4b-4d44-86dc-3d035523ebc6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:30:44.235505 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066cc662-7f4b-4d44-86dc-3d035523ebc6 service neutron] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:30:44.246612 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1389/2784] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:30:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:30:44.489224 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ff9d64-c78e-4646-b183-87f3f6284dd7 None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:30:44.490810 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-b8ff9d64-c78e-4646-b183-87f3f6284dd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:30:44.506297 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ff9d64-c78e-4646-b183-87f3f6284dd7 None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:30:44.519267 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63453189-b988-447c-8678-40c1e670d3fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:30:44.520105 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63453189-b988-447c-8678-40c1e670d3fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:44.520481 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63453189-b988-447c-8678-40c1e670d3fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:44.521048 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63453189-b988-447c-8678-40c1e670d3fe admin admin] PATH_INFO: `/v3/users/0c58e96ca245457aadcb6ee9caac5b30` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:44.522464 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63453189-b988-447c-8678-40c1e670d3fe admin admin] RBAC: Authorizing `identity:delete_user(user_id=0c58e96ca245457aadcb6ee9caac5b30)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:30:44.524868 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63453189-b988-447c-8678-40c1e670d3fe admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:30:44.782919 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-63453189-b988-447c-8678-40c1e670d3fe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0c58e96ca245457aadcb6ee9caac5b30'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:44.811620 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=90,keystone:INSERT=1,keystone:DELETE=9 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:30:44.813171 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-63453189-b988-447c-8678-40c1e670d3fe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0c58e96ca245457aadcb6ee9caac5b30'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:44.814722 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-63453189-b988-447c-8678-40c1e670d3fe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0c58e96ca245457aadcb6ee9caac5b30'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:44.828251 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-63453189-b988-447c-8678-40c1e670d3fe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0c58e96ca245457aadcb6ee9caac5b30'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:44.847124 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066cc662-7f4b-4d44-86dc-3d035523ebc6 service neutron] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:30:44.848846 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-066cc662-7f4b-4d44-86dc-3d035523ebc6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:30:44.862814 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066cc662-7f4b-4d44-86dc-3d035523ebc6 service neutron] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:30:44.870016 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb984c9-c2ba-46ef-a138-7bf65bfee9ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:30:44.870667 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb984c9-c2ba-46ef-a138-7bf65bfee9ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:44.870940 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb984c9-c2ba-46ef-a138-7bf65bfee9ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:44.875970 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb984c9-c2ba-46ef-a138-7bf65bfee9ef admin admin] PATH_INFO: `/v3/projects/600c8e990f9347debe7491f19214f555` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:44.875970 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb984c9-c2ba-46ef-a138-7bf65bfee9ef admin admin] RBAC: Authorizing `identity:delete_project(project_id=600c8e990f9347debe7491f19214f555)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:30:44.875970 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb984c9-c2ba-46ef-a138-7bf65bfee9ef admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:30:44.902524 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-63453189-b988-447c-8678-40c1e670d3fe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0c58e96ca245457aadcb6ee9caac5b30'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:44.917686 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1396/2785] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 01:30:44 2022] DELETE /identity/v3/users/0c58e96ca245457aadcb6ee9caac5b30 => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:30:44.928014 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e922bf-c284-4d9d-b3b2-8547fe270a50 None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:44.928014 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e922bf-c284-4d9d-b3b2-8547fe270a50 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:44.928014 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e922bf-c284-4d9d-b3b2-8547fe270a50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:44.947825 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-47e922bf-c284-4d9d-b3b2-8547fe270a50 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:30:44.949074 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-47e922bf-c284-4d9d-b3b2-8547fe270a50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:30:45.010706 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1397/2786] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:30:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:30:45.020748 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e922bf-c284-4d9d-b3b2-8547fe270a50 None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:30:45.022914 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-47e922bf-c284-4d9d-b3b2-8547fe270a50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:30:45.031176 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-9fb984c9-c2ba-46ef-a138-7bf65bfee9ef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '600c8e990f9347debe7491f19214f555', 'request_id': 'req-9fb984c9-c2ba-46ef-a138-7bf65bfee9ef'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:45.038278 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-9fb984c9-c2ba-46ef-a138-7bf65bfee9ef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '600c8e990f9347debe7491f19214f555', 'request_id': 'req-9fb984c9-c2ba-46ef-a138-7bf65bfee9ef'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:45.045227 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-9fb984c9-c2ba-46ef-a138-7bf65bfee9ef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '600c8e990f9347debe7491f19214f555', 'request_id': 'req-9fb984c9-c2ba-46ef-a138-7bf65bfee9ef'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:45.085606 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e922bf-c284-4d9d-b3b2-8547fe270a50 None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:30:45.123827 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-9fb984c9-c2ba-46ef-a138-7bf65bfee9ef admin admin] The token cache is being invalidate because project 600c8e990f9347debe7491f19214f555 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81749) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 01:30:45.124408 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-9fb984c9-c2ba-46ef-a138-7bf65bfee9ef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:45.125207 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-9fb984c9-c2ba-46ef-a138-7bf65bfee9ef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:45.127052 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1390/2787] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 01:30:44 2022] DELETE /identity/v3/projects/600c8e990f9347debe7491f19214f555 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 01:30:45.132216 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fb3d7a-7380-4c1e-be40-47f88542a4c4 None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:45.132784 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fb3d7a-7380-4c1e-be40-47f88542a4c4 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:45.133071 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fb3d7a-7380-4c1e-be40-47f88542a4c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:45.135922 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9babcb-67f8-4b3f-b334-4a677bed7d65 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0817c3320d5d41d8bba0550c8ae8dd60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:30:45.136592 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9babcb-67f8-4b3f-b334-4a677bed7d65 service neutron] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:45.136916 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9babcb-67f8-4b3f-b334-4a677bed7d65 service neutron] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:45.137190 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9babcb-67f8-4b3f-b334-4a677bed7d65 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:45.140131 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-ab9babcb-67f8-4b3f-b334-4a677bed7d65 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:30:45.155075 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-b8fb3d7a-7380-4c1e-be40-47f88542a4c4 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:30:45.157500 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-b8fb3d7a-7380-4c1e-be40-47f88542a4c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:30:45.176686 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:30:45.215690 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:30:45.231179 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9babcb-67f8-4b3f-b334-4a677bed7d65 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:30:45.232659 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9babcb-67f8-4b3f-b334-4a677bed7d65 service neutron] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:30:45.234552 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1391/2788] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:30:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 01:30:45.235641 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fb3d7a-7380-4c1e-be40-47f88542a4c4 None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:30:45.237070 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-b8fb3d7a-7380-4c1e-be40-47f88542a4c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:30:45.243397 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1398/2789] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:30:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 01:30:45.245258 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaa1f2b-117f-44a6-b09e-96c172fe633e None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:45.245258 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaa1f2b-117f-44a6-b09e-96c172fe633e None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:45.245258 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaa1f2b-117f-44a6-b09e-96c172fe633e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:45.272069 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-ebaa1f2b-117f-44a6-b09e-96c172fe633e None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:30:45.273720 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-ebaa1f2b-117f-44a6-b09e-96c172fe633e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:30:45.288275 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1399/2790] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:30:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:30:45.292537 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebaa1f2b-117f-44a6-b09e-96c172fe633e None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:30:45.304668 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebaa1f2b-117f-44a6-b09e-96c172fe633e None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:30:45.312633 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813cbbd5-b765-4198-8f02-49e9debe96e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:30:45.313682 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813cbbd5-b765-4198-8f02-49e9debe96e0 admin admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:45.314434 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813cbbd5-b765-4198-8f02-49e9debe96e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:45.314925 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813cbbd5-b765-4198-8f02-49e9debe96e0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:45.315964 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813cbbd5-b765-4198-8f02-49e9debe96e0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:30:45.315964 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813cbbd5-b765-4198-8f02-49e9debe96e0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:30:45.317798 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813cbbd5-b765-4198-8f02-49e9debe96e0 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:30:45.332794 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1400/2791] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Aug 31 01:30:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 01:30:45.336829 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813cbbd5-b765-4198-8f02-49e9debe96e0 admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:30:45.347238 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813cbbd5-b765-4198-8f02-49e9debe96e0 admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:30:45.354760 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1aa58c-c3db-435b-bb42-111e1d186e91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:30:45.355697 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1aa58c-c3db-435b-bb42-111e1d186e91 admin admin] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:45.356710 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1aa58c-c3db-435b-bb42-111e1d186e91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:45.357032 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1aa58c-c3db-435b-bb42-111e1d186e91 admin admin] PATH_INFO: `/v3/projects` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:45.357854 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1aa58c-c3db-435b-bb42-111e1d186e91 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:30:45.359940 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1aa58c-c3db-435b-bb42-111e1d186e91 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:30:45.366293 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fb3d7a-7380-4c1e-be40-47f88542a4c4 None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:30:45.373588 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3391d59-4d64-4494-b0e0-6fb12fb22b08 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dea0fe5b1d6b405cb47b95192580303b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:30:45.374432 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3391d59-4d64-4494-b0e0-6fb12fb22b08 service placement] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:45.375375 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3391d59-4d64-4494-b0e0-6fb12fb22b08 service placement] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:45.375656 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3391d59-4d64-4494-b0e0-6fb12fb22b08 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:45.377506 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-d3391d59-4d64-4494-b0e0-6fb12fb22b08 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:30:45.391312 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3391d59-4d64-4494-b0e0-6fb12fb22b08 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:30:45.393208 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3391d59-4d64-4494-b0e0-6fb12fb22b08 service placement] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:30:45.403657 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1392/2792] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:30:45 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:30:45.456995 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1401/2793] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 01:30:45 2022] POST /identity/v3/projects => generated 373 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:30:45.461732 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3391d59-4d64-4494-b0e0-6fb12fb22b08 service placement] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:30:45.473377 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3391d59-4d64-4494-b0e0-6fb12fb22b08 service placement] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:30:45.552765 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb863ba4-9e1a-4171-8682-4a61bb3dff0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:30:45.552765 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb863ba4-9e1a-4171-8682-4a61bb3dff0f admin admin] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:45.552765 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb863ba4-9e1a-4171-8682-4a61bb3dff0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:45.552765 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb863ba4-9e1a-4171-8682-4a61bb3dff0f admin admin] PATH_INFO: `/v3/users` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:45.553223 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb863ba4-9e1a-4171-8682-4a61bb3dff0f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:30:45.554933 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb863ba4-9e1a-4171-8682-4a61bb3dff0f admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:30:45.711928 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1393/2794] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 01:30:45 2022] POST /identity/v3/users => generated 287 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 01:30:45.717236 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1aa58c-c3db-435b-bb42-111e1d186e91 admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:30:45.733455 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1aa58c-c3db-435b-bb42-111e1d186e91 admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:30:45.761336 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9992d744-ff06-48f5-ba52-848a83b242fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:30:45.764960 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9992d744-ff06-48f5-ba52-848a83b242fd admin admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:45.765252 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9992d744-ff06-48f5-ba52-848a83b242fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:45.765574 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9992d744-ff06-48f5-ba52-848a83b242fd admin admin] PATH_INFO: `/v3/roles` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:45.766361 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9992d744-ff06-48f5-ba52-848a83b242fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:30:45.768752 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9992d744-ff06-48f5-ba52-848a83b242fd admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:30:45.779974 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb863ba4-9e1a-4171-8682-4a61bb3dff0f admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:30:45.779974 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-bb863ba4-9e1a-4171-8682-4a61bb3dff0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:30:45.783564 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1402/2795] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:30:45 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:30:45.787486 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9992d744-ff06-48f5-ba52-848a83b242fd admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:30:45.803949 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9992d744-ff06-48f5-ba52-848a83b242fd admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:30:45.824787 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb863ba4-9e1a-4171-8682-4a61bb3dff0f admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:30:45.832477 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce87e7b-0de2-436b-93eb-36be05f3fb8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:30:45.833808 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce87e7b-0de2-436b-93eb-36be05f3fb8b admin admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:45.834794 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce87e7b-0de2-436b-93eb-36be05f3fb8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:45.834949 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce87e7b-0de2-436b-93eb-36be05f3fb8b admin admin] PATH_INFO: `/v3/roles` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:45.836248 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce87e7b-0de2-436b-93eb-36be05f3fb8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:30:45.838888 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce87e7b-0de2-436b-93eb-36be05f3fb8b admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:30:45.839819 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15200fe5-689e-4f66-919e-574384004f68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:30:45.840569 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15200fe5-689e-4f66-919e-574384004f68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:45.840974 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15200fe5-689e-4f66-919e-574384004f68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:45.841340 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15200fe5-689e-4f66-919e-574384004f68 admin admin] PATH_INFO: `/v3/projects/541dad74c8a7404f969a872cc18e4dd8` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:45.844601 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15200fe5-689e-4f66-919e-574384004f68 admin admin] RBAC: Authorizing `identity:delete_project(project_id=541dad74c8a7404f969a872cc18e4dd8)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:30:45.850477 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15200fe5-689e-4f66-919e-574384004f68 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:30:45.868641 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1403/2796] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:30:45 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:30:45.871801 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce87e7b-0de2-436b-93eb-36be05f3fb8b admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:30:45.892220 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:30:45.894416 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce87e7b-0de2-436b-93eb-36be05f3fb8b admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:30:45.913031 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6881ec6-1615-4514-abc9-c4d7513f50aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:30:45.913739 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6881ec6-1615-4514-abc9-c4d7513f50aa admin admin] REQUEST_METHOD: `PUT` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:45.914026 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6881ec6-1615-4514-abc9-c4d7513f50aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:45.914305 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6881ec6-1615-4514-abc9-c4d7513f50aa admin admin] PATH_INFO: `/v3/projects/868b605d70a24a35a1d8ac0c23cb51d6/users/4a2301bffac547de94cda347fcd68b0e/roles/d0b9768f4d014766b9326387f2e11f34` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:45.932932 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6881ec6-1615-4514-abc9-c4d7513f50aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=868b605d70a24a35a1d8ac0c23cb51d6, user_id=4a2301bffac547de94cda347fcd68b0e, role_id=d0b9768f4d014766b9326387f2e11f34)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:30:45.936166 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6881ec6-1615-4514-abc9-c4d7513f50aa admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:30:46.004123 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1404/2797] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 01:30:45 2022] PUT /identity/v3/projects/868b605d70a24a35a1d8ac0c23cb51d6/users/4a2301bffac547de94cda347fcd68b0e/roles/d0b9768f4d014766b9326387f2e11f34 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:30:46.018287 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dbfdba-843f-4274-b32e-a00a1a9185c3 None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:46.018581 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dbfdba-843f-4274-b32e-a00a1a9185c3 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:46.023375 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dbfdba-843f-4274-b32e-a00a1a9185c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:46.028777 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-15200fe5-689e-4f66-919e-574384004f68 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '541dad74c8a7404f969a872cc18e4dd8', 'request_id': 'req-15200fe5-689e-4f66-919e-574384004f68'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:46.052267 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-15200fe5-689e-4f66-919e-574384004f68 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '541dad74c8a7404f969a872cc18e4dd8', 'request_id': 'req-15200fe5-689e-4f66-919e-574384004f68'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:46.064708 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-15200fe5-689e-4f66-919e-574384004f68 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '541dad74c8a7404f969a872cc18e4dd8', 'request_id': 'req-15200fe5-689e-4f66-919e-574384004f68'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:46.071609 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-00dbfdba-843f-4274-b32e-a00a1a9185c3 None None] MFA Rules not processed for user `4a2301bffac547de94cda347fcd68b0e`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:30:46.076109 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-00dbfdba-843f-4274-b32e-a00a1a9185c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:30:46.151236 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-15200fe5-689e-4f66-919e-574384004f68 admin admin] The token cache is being invalidate because project 541dad74c8a7404f969a872cc18e4dd8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81749) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 01:30:46.153299 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-15200fe5-689e-4f66-919e-574384004f68 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:46.153299 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-15200fe5-689e-4f66-919e-574384004f68 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:46.159775 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1394/2798] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 01:30:45 2022] DELETE /identity/v3/projects/541dad74c8a7404f969a872cc18e4dd8 => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:30:46.161133 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15200fe5-689e-4f66-919e-574384004f68 admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:30:46.164105 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-15200fe5-689e-4f66-919e-574384004f68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:30:46.230927 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1405/2799] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:30:46 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:30:46.252308 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dbfdba-843f-4274-b32e-a00a1a9185c3 None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:30:46.254033 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-00dbfdba-843f-4274-b32e-a00a1a9185c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:30:46.276358 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15200fe5-689e-4f66-919e-574384004f68 admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:30:46.283369 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d16df3-0e0a-4f8f-92b0-d2a54f057195 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:30:46.284022 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d16df3-0e0a-4f8f-92b0-d2a54f057195 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:46.284307 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d16df3-0e0a-4f8f-92b0-d2a54f057195 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:46.287662 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d16df3-0e0a-4f8f-92b0-d2a54f057195 admin admin] PATH_INFO: `/v3/users/2fefd987a5da4fa59cb6464aa30a2470` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:46.287662 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d16df3-0e0a-4f8f-92b0-d2a54f057195 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2fefd987a5da4fa59cb6464aa30a2470)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:30:46.288096 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d16df3-0e0a-4f8f-92b0-d2a54f057195 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:30:46.334615 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:30:46.339910 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dbfdba-843f-4274-b32e-a00a1a9185c3 None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:30:46.343960 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cf66b3-cb0a-477a-ac66-e0d516aa3326 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5c8fc69c518412ba1aad61a9f4f558c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:30:46.344584 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cf66b3-cb0a-477a-ac66-e0d516aa3326 service cinder] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:46.344901 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cf66b3-cb0a-477a-ac66-e0d516aa3326 service cinder] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:46.345180 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cf66b3-cb0a-477a-ac66-e0d516aa3326 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:46.356237 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cf66b3-cb0a-477a-ac66-e0d516aa3326 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:30:46.358685 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cf66b3-cb0a-477a-ac66-e0d516aa3326 service cinder] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:30:46.370056 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1406/2800] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:30:46 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 01:30:46.569823 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-69d16df3-0e0a-4f8f-92b0-d2a54f057195 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2fefd987a5da4fa59cb6464aa30a2470'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:46.666292 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-69d16df3-0e0a-4f8f-92b0-d2a54f057195 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2fefd987a5da4fa59cb6464aa30a2470'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:46.671982 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-69d16df3-0e0a-4f8f-92b0-d2a54f057195 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2fefd987a5da4fa59cb6464aa30a2470'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:46.676593 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-69d16df3-0e0a-4f8f-92b0-d2a54f057195 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2fefd987a5da4fa59cb6464aa30a2470'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:46.677458 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-69d16df3-0e0a-4f8f-92b0-d2a54f057195 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2fefd987a5da4fa59cb6464aa30a2470'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:46.705463 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1395/2801] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 01:30:46 2022] DELETE /identity/v3/users/2fefd987a5da4fa59cb6464aa30a2470 => generated 0 bytes in 545 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:30:47.031505 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fd4f0d-e467-4196-ae11-992f19f95f98 None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:47.031987 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fd4f0d-e467-4196-ae11-992f19f95f98 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:47.035158 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fd4f0d-e467-4196-ae11-992f19f95f98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:47.086776 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-87fd4f0d-e467-4196-ae11-992f19f95f98 None None] MFA Rules not processed for user `b3efb4f9daac4a0ca05a36dd361f17e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:30:47.088796 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-87fd4f0d-e467-4196-ae11-992f19f95f98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:30:47.198321 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1407/2802] 10.222.0.20 () {58 vars in 967 bytes} [Wed Aug 31 01:30:47 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 01:30:47.254687 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d16df3-0e0a-4f8f-92b0-d2a54f057195 admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:30:47.261199 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-69d16df3-0e0a-4f8f-92b0-d2a54f057195 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:30:47.292035 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:30:47.357086 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d16df3-0e0a-4f8f-92b0-d2a54f057195 admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:30:47.362345 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fd4f0d-e467-4196-ae11-992f19f95f98 None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:30:47.364033 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-87fd4f0d-e467-4196-ae11-992f19f95f98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:30:47.364401 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a10b7fd-4be5-4390-bbc3-56101736f446 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:30:47.365074 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a10b7fd-4be5-4390-bbc3-56101736f446 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:47.365509 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a10b7fd-4be5-4390-bbc3-56101736f446 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:47.366472 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a10b7fd-4be5-4390-bbc3-56101736f446 admin admin] PATH_INFO: `/v3/projects/882072fe98cc4ca09c0f8859419af6b8` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:47.367928 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a10b7fd-4be5-4390-bbc3-56101736f446 admin admin] RBAC: Authorizing `identity:delete_project(project_id=882072fe98cc4ca09c0f8859419af6b8)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:30:47.370435 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a10b7fd-4be5-4390-bbc3-56101736f446 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:30:47.466218 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fd4f0d-e467-4196-ae11-992f19f95f98 None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:30:47.471728 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8ebcd8-cd7e-45f8-9466-86b4f43d8a1a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc1d010919ec40878ae829cc8dceec4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:30:47.472417 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8ebcd8-cd7e-45f8-9466-86b4f43d8a1a service swift] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:47.472702 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8ebcd8-cd7e-45f8-9466-86b4f43d8a1a service swift] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:47.473010 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8ebcd8-cd7e-45f8-9466-86b4f43d8a1a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:47.481832 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8ebcd8-cd7e-45f8-9466-86b4f43d8a1a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:30:47.483543 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8ebcd8-cd7e-45f8-9466-86b4f43d8a1a service swift] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:30:47.494665 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1408/2803] 10.222.0.20 () {60 vars in 1384 bytes} [Wed Aug 31 01:30:47 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 31 01:30:47.508345 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-1a10b7fd-4be5-4390-bbc3-56101736f446 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '882072fe98cc4ca09c0f8859419af6b8', 'request_id': 'req-1a10b7fd-4be5-4390-bbc3-56101736f446'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:47.519652 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-1a10b7fd-4be5-4390-bbc3-56101736f446 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '882072fe98cc4ca09c0f8859419af6b8', 'request_id': 'req-1a10b7fd-4be5-4390-bbc3-56101736f446'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:47.525755 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-1a10b7fd-4be5-4390-bbc3-56101736f446 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '882072fe98cc4ca09c0f8859419af6b8', 'request_id': 'req-1a10b7fd-4be5-4390-bbc3-56101736f446'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:47.582018 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-1a10b7fd-4be5-4390-bbc3-56101736f446 admin admin] The token cache is being invalidate because project 882072fe98cc4ca09c0f8859419af6b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81749) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 01:30:47.582312 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-1a10b7fd-4be5-4390-bbc3-56101736f446 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:47.583083 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-1a10b7fd-4be5-4390-bbc3-56101736f446 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:47.585517 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1396/2804] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 01:30:47 2022] DELETE /identity/v3/projects/882072fe98cc4ca09c0f8859419af6b8 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:30:47.589881 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8ebcd8-cd7e-45f8-9466-86b4f43d8a1a service swift] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:30:47.603312 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8ebcd8-cd7e-45f8-9466-86b4f43d8a1a service swift] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:30:47.668242 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b366c3-da2c-4c2e-95b4-86856ca4bfb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:30:47.668925 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b366c3-da2c-4c2e-95b4-86856ca4bfb9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:47.669207 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b366c3-da2c-4c2e-95b4-86856ca4bfb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:47.669480 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b366c3-da2c-4c2e-95b4-86856ca4bfb9 admin admin] PATH_INFO: `/v3/users/82db35e7245240eb8e4af8b288fc42ea` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:47.671219 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b366c3-da2c-4c2e-95b4-86856ca4bfb9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=82db35e7245240eb8e4af8b288fc42ea)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:30:47.673167 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b366c3-da2c-4c2e-95b4-86856ca4bfb9 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:30:47.687533 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:30:47.920389 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-91b366c3-da2c-4c2e-95b4-86856ca4bfb9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '82db35e7245240eb8e4af8b288fc42ea'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:47.933161 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-91b366c3-da2c-4c2e-95b4-86856ca4bfb9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '82db35e7245240eb8e4af8b288fc42ea'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:47.933940 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-91b366c3-da2c-4c2e-95b4-86856ca4bfb9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '82db35e7245240eb8e4af8b288fc42ea'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:47.942320 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-91b366c3-da2c-4c2e-95b4-86856ca4bfb9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '82db35e7245240eb8e4af8b288fc42ea'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:48.032103 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-91b366c3-da2c-4c2e-95b4-86856ca4bfb9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '82db35e7245240eb8e4af8b288fc42ea'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:48.039311 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1409/2805] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 01:30:47 2022] DELETE /identity/v3/users/82db35e7245240eb8e4af8b288fc42ea => generated 0 bytes in 450 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:30:48.697345 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a10b7fd-4be5-4390-bbc3-56101736f446 admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:30:48.699087 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-1a10b7fd-4be5-4390-bbc3-56101736f446 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:30:48.715176 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a10b7fd-4be5-4390-bbc3-56101736f446 admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:30:48.783452 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0013b0f-1191-4794-a4c1-20a566997de6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:30:48.784625 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0013b0f-1191-4794-a4c1-20a566997de6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:48.785180 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0013b0f-1191-4794-a4c1-20a566997de6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:48.785713 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0013b0f-1191-4794-a4c1-20a566997de6 admin admin] PATH_INFO: `/v3/projects/ea06efa087214c1eb593aae0cdf95154` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:48.787018 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0013b0f-1191-4794-a4c1-20a566997de6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ea06efa087214c1eb593aae0cdf95154)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:30:48.789032 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0013b0f-1191-4794-a4c1-20a566997de6 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:30:48.935711 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-b0013b0f-1191-4794-a4c1-20a566997de6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ea06efa087214c1eb593aae0cdf95154', 'request_id': 'req-b0013b0f-1191-4794-a4c1-20a566997de6'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:48.938495 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:30:48.945739 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-b0013b0f-1191-4794-a4c1-20a566997de6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ea06efa087214c1eb593aae0cdf95154', 'request_id': 'req-b0013b0f-1191-4794-a4c1-20a566997de6'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:48.949709 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-b0013b0f-1191-4794-a4c1-20a566997de6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ea06efa087214c1eb593aae0cdf95154', 'request_id': 'req-b0013b0f-1191-4794-a4c1-20a566997de6'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:49.016464 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-b0013b0f-1191-4794-a4c1-20a566997de6 admin admin] The token cache is being invalidate because project ea06efa087214c1eb593aae0cdf95154 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81749) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 01:30:49.017093 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-b0013b0f-1191-4794-a4c1-20a566997de6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:49.018959 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-b0013b0f-1191-4794-a4c1-20a566997de6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:49.020803 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1397/2806] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 01:30:48 2022] DELETE /identity/v3/projects/ea06efa087214c1eb593aae0cdf95154 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:30:49.028080 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5d3935-ead7-4dde-9ef9-798b7a43bbf2 None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:49.028677 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5d3935-ead7-4dde-9ef9-798b7a43bbf2 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:49.030148 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5d3935-ead7-4dde-9ef9-798b7a43bbf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:49.064064 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-3c5d3935-ead7-4dde-9ef9-798b7a43bbf2 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:30:49.065316 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-3c5d3935-ead7-4dde-9ef9-798b7a43bbf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:30:49.144608 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1410/2807] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:30:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:30:49.152435 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdfde1e-16db-4282-8cc5-2b3ee1fc24d9 None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:49.152753 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdfde1e-16db-4282-8cc5-2b3ee1fc24d9 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:49.153140 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdfde1e-16db-4282-8cc5-2b3ee1fc24d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:49.179961 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-9fdfde1e-16db-4282-8cc5-2b3ee1fc24d9 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:30:49.181711 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-9fdfde1e-16db-4282-8cc5-2b3ee1fc24d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:30:49.190815 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1398/2808] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:30:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:30:49.195471 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5d3935-ead7-4dde-9ef9-798b7a43bbf2 None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:30:49.206969 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5d3935-ead7-4dde-9ef9-798b7a43bbf2 None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:30:49.212026 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea25807-1c37-419a-a035-b1bb6c4cdcff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:30:49.212652 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea25807-1c37-419a-a035-b1bb6c4cdcff admin admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:49.213135 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea25807-1c37-419a-a035-b1bb6c4cdcff admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:49.213406 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea25807-1c37-419a-a035-b1bb6c4cdcff admin admin] PATH_INFO: `/v3/domains` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:49.214067 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea25807-1c37-419a-a035-b1bb6c4cdcff admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:30:49.214368 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea25807-1c37-419a-a035-b1bb6c4cdcff admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:30:49.217494 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea25807-1c37-419a-a035-b1bb6c4cdcff admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:30:49.233006 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1411/2809] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Aug 31 01:30:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 01:30:49.237908 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdfde1e-16db-4282-8cc5-2b3ee1fc24d9 None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:30:49.249169 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdfde1e-16db-4282-8cc5-2b3ee1fc24d9 None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:30:49.258268 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e21bc7-6f85-4cff-a8aa-84e7462f7417 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:30:49.265399 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e21bc7-6f85-4cff-a8aa-84e7462f7417 admin admin] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:49.265399 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e21bc7-6f85-4cff-a8aa-84e7462f7417 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:49.265399 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e21bc7-6f85-4cff-a8aa-84e7462f7417 admin admin] PATH_INFO: `/v3/projects` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:49.265399 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e21bc7-6f85-4cff-a8aa-84e7462f7417 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:30:49.265399 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e21bc7-6f85-4cff-a8aa-84e7462f7417 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:30:49.414078 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1399/2810] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 01:30:49 2022] POST /identity/v3/projects => generated 373 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:30:49.419470 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea25807-1c37-419a-a035-b1bb6c4cdcff admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:30:49.441627 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea25807-1c37-419a-a035-b1bb6c4cdcff admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:30:49.546250 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f93d7c4-b990-49d8-8143-e3a56986bf52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:30:49.547124 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f93d7c4-b990-49d8-8143-e3a56986bf52 admin admin] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:49.547632 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f93d7c4-b990-49d8-8143-e3a56986bf52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:49.547886 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f93d7c4-b990-49d8-8143-e3a56986bf52 admin admin] PATH_INFO: `/v3/users` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:49.548564 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f93d7c4-b990-49d8-8143-e3a56986bf52 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:30:49.550529 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f93d7c4-b990-49d8-8143-e3a56986bf52 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:30:49.685291 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1412/2811] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 01:30:49 2022] POST /identity/v3/users => generated 287 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:30:49.689503 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e21bc7-6f85-4cff-a8aa-84e7462f7417 admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:30:49.700014 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e21bc7-6f85-4cff-a8aa-84e7462f7417 admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:30:49.705092 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7309633-f03f-4a87-b16c-4937e015a125 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:30:49.706025 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7309633-f03f-4a87-b16c-4937e015a125 admin admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:49.706380 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7309633-f03f-4a87-b16c-4937e015a125 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:49.706824 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7309633-f03f-4a87-b16c-4937e015a125 admin admin] PATH_INFO: `/v3/roles` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:49.707543 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7309633-f03f-4a87-b16c-4937e015a125 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:30:49.709262 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7309633-f03f-4a87-b16c-4937e015a125 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:30:49.720351 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1400/2812] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:30:49 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:30:49.724381 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f93d7c4-b990-49d8-8143-e3a56986bf52 admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:30:49.735801 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f93d7c4-b990-49d8-8143-e3a56986bf52 admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:30:49.741040 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850d26cb-8c2c-4a86-ae61-573982ddbfc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:30:49.741798 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850d26cb-8c2c-4a86-ae61-573982ddbfc8 admin admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:49.743744 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850d26cb-8c2c-4a86-ae61-573982ddbfc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:49.744133 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850d26cb-8c2c-4a86-ae61-573982ddbfc8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:49.744926 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850d26cb-8c2c-4a86-ae61-573982ddbfc8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:30:49.746678 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850d26cb-8c2c-4a86-ae61-573982ddbfc8 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:30:49.748504 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:30:49.761250 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1413/2813] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:30:49 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:30:49.765797 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7309633-f03f-4a87-b16c-4937e015a125 admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:30:49.778298 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7309633-f03f-4a87-b16c-4937e015a125 admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:30:49.788618 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b545195-b25e-4990-a12f-e22cbbd00408 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:30:49.789248 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b545195-b25e-4990-a12f-e22cbbd00408 admin admin] REQUEST_METHOD: `PUT` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:49.789533 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b545195-b25e-4990-a12f-e22cbbd00408 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:49.789929 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b545195-b25e-4990-a12f-e22cbbd00408 admin admin] PATH_INFO: `/v3/projects/f9084d8c977147b7a3652e34466b2522/users/78b0a7bb7fed432cadfdd79c2ddab8c4/roles/d0b9768f4d014766b9326387f2e11f34` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:49.811064 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b545195-b25e-4990-a12f-e22cbbd00408 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9084d8c977147b7a3652e34466b2522, user_id=78b0a7bb7fed432cadfdd79c2ddab8c4, role_id=d0b9768f4d014766b9326387f2e11f34)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:30:49.814133 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b545195-b25e-4990-a12f-e22cbbd00408 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:30:49.900300 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1401/2814] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 01:30:49 2022] PUT /identity/v3/projects/f9084d8c977147b7a3652e34466b2522/users/78b0a7bb7fed432cadfdd79c2ddab8c4/roles/d0b9768f4d014766b9326387f2e11f34 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:30:49.906554 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf2d9b0-9637-43b4-94f2-571da83ac05f None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:49.906943 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf2d9b0-9637-43b4-94f2-571da83ac05f None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:49.907316 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf2d9b0-9637-43b4-94f2-571da83ac05f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:49.928837 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-2cf2d9b0-9637-43b4-94f2-571da83ac05f None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:30:49.938188 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-2cf2d9b0-9637-43b4-94f2-571da83ac05f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:30:50.010986 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1414/2815] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:30:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 01:30:50.026034 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b545195-b25e-4990-a12f-e22cbbd00408 admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:30:50.027922 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-8b545195-b25e-4990-a12f-e22cbbd00408 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:30:50.143223 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b545195-b25e-4990-a12f-e22cbbd00408 admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:30:50.147253 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7f5bc8-d929-4d25-96b0-c9702f062d43 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0817c3320d5d41d8bba0550c8ae8dd60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:30:50.147862 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7f5bc8-d929-4d25-96b0-c9702f062d43 service neutron] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:50.148654 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7f5bc8-d929-4d25-96b0-c9702f062d43 service neutron] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:50.148654 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7f5bc8-d929-4d25-96b0-c9702f062d43 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:50.159749 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7f5bc8-d929-4d25-96b0-c9702f062d43 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:30:50.161250 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7f5bc8-d929-4d25-96b0-c9702f062d43 service neutron] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:30:50.172065 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1402/2816] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:30:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:30:50.878912 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf2d9b0-9637-43b4-94f2-571da83ac05f None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:30:50.880392 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-2cf2d9b0-9637-43b4-94f2-571da83ac05f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:30:50.902567 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf2d9b0-9637-43b4-94f2-571da83ac05f None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:30:50.907244 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89568f8f-6537-472d-9fe0-e9bb831d0546 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:30:50.908495 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89568f8f-6537-472d-9fe0-e9bb831d0546 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:50.908927 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89568f8f-6537-472d-9fe0-e9bb831d0546 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:50.909299 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89568f8f-6537-472d-9fe0-e9bb831d0546 admin admin] PATH_INFO: `/v3/users/4a2301bffac547de94cda347fcd68b0e` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:50.910599 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89568f8f-6537-472d-9fe0-e9bb831d0546 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4a2301bffac547de94cda347fcd68b0e)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:30:50.912751 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89568f8f-6537-472d-9fe0-e9bb831d0546 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:30:51.058105 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-89568f8f-6537-472d-9fe0-e9bb831d0546 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4a2301bffac547de94cda347fcd68b0e'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:51.094313 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-89568f8f-6537-472d-9fe0-e9bb831d0546 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4a2301bffac547de94cda347fcd68b0e'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:51.095355 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-89568f8f-6537-472d-9fe0-e9bb831d0546 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4a2301bffac547de94cda347fcd68b0e'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:51.111149 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-89568f8f-6537-472d-9fe0-e9bb831d0546 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4a2301bffac547de94cda347fcd68b0e'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:51.221092 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-89568f8f-6537-472d-9fe0-e9bb831d0546 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4a2301bffac547de94cda347fcd68b0e'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:51.231361 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1415/2817] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 01:30:50 2022] DELETE /identity/v3/users/4a2301bffac547de94cda347fcd68b0e => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:30:51.233868 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25066a5c-5d57-4b91-b7f5-e6a07d76a8fa None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:51.234541 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25066a5c-5d57-4b91-b7f5-e6a07d76a8fa None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:51.234849 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25066a5c-5d57-4b91-b7f5-e6a07d76a8fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:51.255953 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-25066a5c-5d57-4b91-b7f5-e6a07d76a8fa None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:30:51.257201 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-25066a5c-5d57-4b91-b7f5-e6a07d76a8fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:30:51.336363 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:30:51.348493 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1403/2818] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:30:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:30:51.365895 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89568f8f-6537-472d-9fe0-e9bb831d0546 admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:30:51.367261 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-89568f8f-6537-472d-9fe0-e9bb831d0546 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:30:51.462097 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89568f8f-6537-472d-9fe0-e9bb831d0546 admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:30:51.470432 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea142af-3802-4513-9000-5ed441c8b32e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0817c3320d5d41d8bba0550c8ae8dd60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:30:51.471066 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea142af-3802-4513-9000-5ed441c8b32e service neutron] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:51.471349 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea142af-3802-4513-9000-5ed441c8b32e service neutron] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:51.471626 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea142af-3802-4513-9000-5ed441c8b32e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:51.484243 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea142af-3802-4513-9000-5ed441c8b32e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:30:51.485990 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea142af-3802-4513-9000-5ed441c8b32e service neutron] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:30:51.499898 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1416/2819] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:30:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:30:51.918906 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0406362e-f752-4227-9dc9-e24d2e1135e1 None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:51.919194 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0406362e-f752-4227-9dc9-e24d2e1135e1 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:51.919473 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0406362e-f752-4227-9dc9-e24d2e1135e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:51.961201 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-0406362e-f752-4227-9dc9-e24d2e1135e1 None None] MFA Rules not processed for user `78b0a7bb7fed432cadfdd79c2ddab8c4`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:30:51.962511 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-0406362e-f752-4227-9dc9-e24d2e1135e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:30:52.048411 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1404/2820] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:30:51 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:30:52.067388 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea142af-3802-4513-9000-5ed441c8b32e service neutron] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:30:52.074302 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-0ea142af-3802-4513-9000-5ed441c8b32e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:30:52.089666 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:30:52.157770 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea142af-3802-4513-9000-5ed441c8b32e service neutron] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:30:52.161521 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30204e4c-0d25-47e3-a45c-847915ad800f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5c8fc69c518412ba1aad61a9f4f558c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:30:52.162192 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30204e4c-0d25-47e3-a45c-847915ad800f service cinder] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:52.162466 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30204e4c-0d25-47e3-a45c-847915ad800f service cinder] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:52.163385 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30204e4c-0d25-47e3-a45c-847915ad800f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:52.172540 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30204e4c-0d25-47e3-a45c-847915ad800f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:30:52.174014 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30204e4c-0d25-47e3-a45c-847915ad800f service cinder] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:30:52.183478 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1417/2821] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:30:52 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:30:52.214702 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0406362e-f752-4227-9dc9-e24d2e1135e1 None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:30:52.217875 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-0406362e-f752-4227-9dc9-e24d2e1135e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:30:52.235869 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0406362e-f752-4227-9dc9-e24d2e1135e1 None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:30:52.241807 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3ef8b7-4503-45ef-9d89-2048dd3508f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:30:52.243288 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3ef8b7-4503-45ef-9d89-2048dd3508f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:52.243979 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3ef8b7-4503-45ef-9d89-2048dd3508f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:52.244884 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3ef8b7-4503-45ef-9d89-2048dd3508f9 admin admin] PATH_INFO: `/v3/projects/868b605d70a24a35a1d8ac0c23cb51d6` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:52.246700 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3ef8b7-4503-45ef-9d89-2048dd3508f9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=868b605d70a24a35a1d8ac0c23cb51d6)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:30:52.250250 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3ef8b7-4503-45ef-9d89-2048dd3508f9 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:30:52.378231 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-3c3ef8b7-4503-45ef-9d89-2048dd3508f9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '868b605d70a24a35a1d8ac0c23cb51d6', 'request_id': 'req-3c3ef8b7-4503-45ef-9d89-2048dd3508f9'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:52.386864 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-3c3ef8b7-4503-45ef-9d89-2048dd3508f9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '868b605d70a24a35a1d8ac0c23cb51d6', 'request_id': 'req-3c3ef8b7-4503-45ef-9d89-2048dd3508f9'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:52.406345 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-3c3ef8b7-4503-45ef-9d89-2048dd3508f9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '868b605d70a24a35a1d8ac0c23cb51d6', 'request_id': 'req-3c3ef8b7-4503-45ef-9d89-2048dd3508f9'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:52.471008 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-3c3ef8b7-4503-45ef-9d89-2048dd3508f9 admin admin] The token cache is being invalidate because project 868b605d70a24a35a1d8ac0c23cb51d6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81749) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 01:30:52.471314 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-3c3ef8b7-4503-45ef-9d89-2048dd3508f9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:52.473848 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-3c3ef8b7-4503-45ef-9d89-2048dd3508f9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:52.473848 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1405/2822] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 01:30:52 2022] DELETE /identity/v3/projects/868b605d70a24a35a1d8ac0c23cb51d6 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:30:52.481286 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4963f93-1c2f-4a61-a829-2bf982089c66 None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:52.481873 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4963f93-1c2f-4a61-a829-2bf982089c66 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:52.482831 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4963f93-1c2f-4a61-a829-2bf982089c66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:52.482958 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3ef8b7-4503-45ef-9d89-2048dd3508f9 admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:30:52.484511 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-3c3ef8b7-4503-45ef-9d89-2048dd3508f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:30:52.514255 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-f4963f93-1c2f-4a61-a829-2bf982089c66 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:30:52.515956 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-f4963f93-1c2f-4a61-a829-2bf982089c66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:30:52.567270 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3ef8b7-4503-45ef-9d89-2048dd3508f9 admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:30:52.571949 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b506f89-d63a-47ff-a69a-e8406c83cf81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:30:52.572571 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b506f89-d63a-47ff-a69a-e8406c83cf81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:52.572894 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b506f89-d63a-47ff-a69a-e8406c83cf81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:52.573150 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b506f89-d63a-47ff-a69a-e8406c83cf81 admin admin] PATH_INFO: `/v3/users/9142f96b5780486389e271ae19423349` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:52.574322 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b506f89-d63a-47ff-a69a-e8406c83cf81 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9142f96b5780486389e271ae19423349)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:30:52.579667 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b506f89-d63a-47ff-a69a-e8406c83cf81 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:30:52.595567 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:30:52.599641 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1418/2823] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:30:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:30:52.609389 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa1e166-5ad7-4501-9ac6-2344b5bced1d None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:52.609686 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa1e166-5ad7-4501-9ac6-2344b5bced1d None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:52.609957 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa1e166-5ad7-4501-9ac6-2344b5bced1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:52.643042 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-cfa1e166-5ad7-4501-9ac6-2344b5bced1d None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:30:52.644426 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-cfa1e166-5ad7-4501-9ac6-2344b5bced1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:30:52.656881 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1419/2824] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:30:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:30:52.658802 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa1e166-5ad7-4501-9ac6-2344b5bced1d None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:30:52.676925 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa1e166-5ad7-4501-9ac6-2344b5bced1d None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:30:52.692097 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb1f52f-5772-4806-a1bf-3e9418db361a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:30:52.692774 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb1f52f-5772-4806-a1bf-3e9418db361a admin admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:52.693046 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb1f52f-5772-4806-a1bf-3e9418db361a admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:52.693573 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb1f52f-5772-4806-a1bf-3e9418db361a admin admin] PATH_INFO: `/v3/domains` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:52.694463 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb1f52f-5772-4806-a1bf-3e9418db361a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81750) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:30:52.694945 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb1f52f-5772-4806-a1bf-3e9418db361a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:30:52.696760 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb1f52f-5772-4806-a1bf-3e9418db361a admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:30:52.741018 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1420/2825] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Aug 31 01:30:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:30:52.743256 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb1f52f-5772-4806-a1bf-3e9418db361a admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:30:52.766575 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb1f52f-5772-4806-a1bf-3e9418db361a admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:30:52.784167 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8920f38a-ac5b-4e20-acf3-8967dbc5012e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:30:52.785536 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8920f38a-ac5b-4e20-acf3-8967dbc5012e admin admin] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:52.785959 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8920f38a-ac5b-4e20-acf3-8967dbc5012e admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:52.786319 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8920f38a-ac5b-4e20-acf3-8967dbc5012e admin admin] PATH_INFO: `/v3/projects` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:52.787085 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8920f38a-ac5b-4e20-acf3-8967dbc5012e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:30:52.789248 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8920f38a-ac5b-4e20-acf3-8967dbc5012e admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:30:52.885473 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-9b506f89-d63a-47ff-a69a-e8406c83cf81 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9142f96b5780486389e271ae19423349'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:52.911118 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1421/2826] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 01:30:52 2022] POST /identity/v3/projects => generated 383 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:30:52.915516 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8920f38a-ac5b-4e20-acf3-8967dbc5012e admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:30:52.989494 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-9b506f89-d63a-47ff-a69a-e8406c83cf81 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9142f96b5780486389e271ae19423349'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:52.995935 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-9b506f89-d63a-47ff-a69a-e8406c83cf81 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9142f96b5780486389e271ae19423349'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:53.003033 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:30:53.012155 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-9b506f89-d63a-47ff-a69a-e8406c83cf81 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9142f96b5780486389e271ae19423349'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:53.013024 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-9b506f89-d63a-47ff-a69a-e8406c83cf81 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9142f96b5780486389e271ae19423349'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:53.016917 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8920f38a-ac5b-4e20-acf3-8967dbc5012e admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:30:53.022172 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc12c46-697b-42ea-b9b5-b437936ce6e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:30:53.022788 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc12c46-697b-42ea-b9b5-b437936ce6e8 admin admin] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:53.023071 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc12c46-697b-42ea-b9b5-b437936ce6e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:53.023350 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc12c46-697b-42ea-b9b5-b437936ce6e8 admin admin] PATH_INFO: `/v3/users` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:53.023986 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc12c46-697b-42ea-b9b5-b437936ce6e8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:30:53.026248 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc12c46-697b-42ea-b9b5-b437936ce6e8 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:30:53.049259 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1406/2827] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 01:30:52 2022] DELETE /identity/v3/users/9142f96b5780486389e271ae19423349 => generated 0 bytes in 569 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:30:53.055415 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dd146b-2579-44e4-8efa-1601df310914 None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:53.055415 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dd146b-2579-44e4-8efa-1601df310914 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:53.055415 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dd146b-2579-44e4-8efa-1601df310914 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:53.107778 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-f3dd146b-2579-44e4-8efa-1601df310914 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:30:53.110550 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-f3dd146b-2579-44e4-8efa-1601df310914 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:30:53.117726 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1422/2828] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 01:30:52 2022] POST /identity/v3/users => generated 292 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:30:53.121411 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc12c46-697b-42ea-b9b5-b437936ce6e8 admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:30:53.123969 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-bfc12c46-697b-42ea-b9b5-b437936ce6e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:30:53.150053 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc12c46-697b-42ea-b9b5-b437936ce6e8 admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:30:53.231292 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e8c759-c798-487c-8228-541761d65dde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:30:53.232482 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e8c759-c798-487c-8228-541761d65dde admin admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:53.232807 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e8c759-c798-487c-8228-541761d65dde admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:53.233088 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e8c759-c798-487c-8228-541761d65dde admin admin] PATH_INFO: `/v3/roles` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:53.233756 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e8c759-c798-487c-8228-541761d65dde admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:30:53.235543 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e8c759-c798-487c-8228-541761d65dde admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:30:53.242574 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1407/2829] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:30:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:30:53.248209 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1423/2830] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:30:53 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:30:53.252130 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dd146b-2579-44e4-8efa-1601df310914 None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:30:53.254481 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e8c759-c798-487c-8228-541761d65dde admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:30:53.255948 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-38e8c759-c798-487c-8228-541761d65dde admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:30:53.262890 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dd146b-2579-44e4-8efa-1601df310914 None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:30:53.268474 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e96ddc5-4566-4fd6-8c63-c4e9427b68f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:30:53.269191 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e96ddc5-4566-4fd6-8c63-c4e9427b68f5 admin admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:53.269674 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e96ddc5-4566-4fd6-8c63-c4e9427b68f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:53.269960 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e96ddc5-4566-4fd6-8c63-c4e9427b68f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:53.270723 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e96ddc5-4566-4fd6-8c63-c4e9427b68f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:30:53.272593 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e96ddc5-4566-4fd6-8c63-c4e9427b68f5 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:30:53.284314 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1408/2831] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:30:53 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:30:53.293740 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e96ddc5-4566-4fd6-8c63-c4e9427b68f5 admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:30:53.301832 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e96ddc5-4566-4fd6-8c63-c4e9427b68f5 admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:30:53.309135 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45c52f2-03f6-4670-a0a2-1174985ec272 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:30:53.309811 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45c52f2-03f6-4670-a0a2-1174985ec272 admin admin] REQUEST_METHOD: `PUT` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:53.310133 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45c52f2-03f6-4670-a0a2-1174985ec272 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:53.310409 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45c52f2-03f6-4670-a0a2-1174985ec272 admin admin] PATH_INFO: `/v3/projects/29c95fae9a51415b9245125f5faedb37/users/f3e944928454456cabf55a6871dd9cf3/roles/d0b9768f4d014766b9326387f2e11f34` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:53.326963 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45c52f2-03f6-4670-a0a2-1174985ec272 admin admin] RBAC: Authorizing `identity:create_grant(project_id=29c95fae9a51415b9245125f5faedb37, user_id=f3e944928454456cabf55a6871dd9cf3, role_id=d0b9768f4d014766b9326387f2e11f34)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:30:53.329701 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45c52f2-03f6-4670-a0a2-1174985ec272 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:30:53.340508 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e8c759-c798-487c-8228-541761d65dde admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:30:53.345404 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa30f536-2665-4caa-a6b4-dced8df3f5e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0817c3320d5d41d8bba0550c8ae8dd60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:30:53.346187 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa30f536-2665-4caa-a6b4-dced8df3f5e2 service neutron] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:53.346560 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa30f536-2665-4caa-a6b4-dced8df3f5e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:53.346981 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa30f536-2665-4caa-a6b4-dced8df3f5e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:53.358576 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa30f536-2665-4caa-a6b4-dced8df3f5e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:30:53.360165 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa30f536-2665-4caa-a6b4-dced8df3f5e2 service neutron] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:30:53.369940 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1424/2832] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:30:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:30:53.386984 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1409/2833] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 01:30:53 2022] PUT /identity/v3/projects/29c95fae9a51415b9245125f5faedb37/users/f3e944928454456cabf55a6871dd9cf3/roles/d0b9768f4d014766b9326387f2e11f34 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:30:53.391982 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cb0499-6f98-4da7-b0b0-6db991465b6c None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:53.392276 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cb0499-6f98-4da7-b0b0-6db991465b6c None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:53.392557 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cb0499-6f98-4da7-b0b0-6db991465b6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:53.413881 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-a5cb0499-6f98-4da7-b0b0-6db991465b6c None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:30:53.415105 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-a5cb0499-6f98-4da7-b0b0-6db991465b6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:30:53.488241 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1425/2834] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:30:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:30:53.497976 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45c52f2-03f6-4670-a0a2-1174985ec272 admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:30:53.500051 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-f45c52f2-03f6-4670-a0a2-1174985ec272 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:30:53.558968 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45c52f2-03f6-4670-a0a2-1174985ec272 admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:30:53.562937 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c87994-9218-42d3-8837-5fa45e2d144b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0817c3320d5d41d8bba0550c8ae8dd60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:30:53.563742 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c87994-9218-42d3-8837-5fa45e2d144b service neutron] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:53.563914 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c87994-9218-42d3-8837-5fa45e2d144b service neutron] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:53.564159 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c87994-9218-42d3-8837-5fa45e2d144b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:53.574001 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c87994-9218-42d3-8837-5fa45e2d144b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:30:53.575442 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c87994-9218-42d3-8837-5fa45e2d144b service neutron] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:30:53.581527 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:30:53.586627 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1410/2835] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:30:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:30:54.144284 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaaf27b-6a4b-4ebb-99dd-753a784f8ae2 None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:54.145208 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaaf27b-6a4b-4ebb-99dd-753a784f8ae2 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:54.145492 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaaf27b-6a4b-4ebb-99dd-753a784f8ae2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:54.168989 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-fdaaf27b-6a4b-4ebb-99dd-753a784f8ae2 None None] MFA Rules not processed for user `78b0a7bb7fed432cadfdd79c2ddab8c4`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:30:54.168989 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-fdaaf27b-6a4b-4ebb-99dd-753a784f8ae2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:30:54.188526 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:30:54.264876 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1426/2836] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:30:54 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:30:54.307548 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c87994-9218-42d3-8837-5fa45e2d144b service neutron] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:30:54.309229 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-54c87994-9218-42d3-8837-5fa45e2d144b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:30:54.325308 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c87994-9218-42d3-8837-5fa45e2d144b service neutron] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:30:54.330108 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58fb533-4cd7-4113-942a-a329b041b560 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:30:54.331199 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58fb533-4cd7-4113-942a-a329b041b560 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:54.331707 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58fb533-4cd7-4113-942a-a329b041b560 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:54.332171 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58fb533-4cd7-4113-942a-a329b041b560 admin admin] PATH_INFO: `/v3/projects/ee99d9b8ed0f4e9297aff9e64fa53ff7` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:54.333555 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58fb533-4cd7-4113-942a-a329b041b560 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ee99d9b8ed0f4e9297aff9e64fa53ff7)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:30:54.335746 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58fb533-4cd7-4113-942a-a329b041b560 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:30:54.474304 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-e58fb533-4cd7-4113-942a-a329b041b560 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ee99d9b8ed0f4e9297aff9e64fa53ff7', 'request_id': 'req-e58fb533-4cd7-4113-942a-a329b041b560'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:54.481895 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-e58fb533-4cd7-4113-942a-a329b041b560 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ee99d9b8ed0f4e9297aff9e64fa53ff7', 'request_id': 'req-e58fb533-4cd7-4113-942a-a329b041b560'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:54.489804 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-e58fb533-4cd7-4113-942a-a329b041b560 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ee99d9b8ed0f4e9297aff9e64fa53ff7', 'request_id': 'req-e58fb533-4cd7-4113-942a-a329b041b560'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:54.544235 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-e58fb533-4cd7-4113-942a-a329b041b560 admin admin] The token cache is being invalidate because project ee99d9b8ed0f4e9297aff9e64fa53ff7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81749) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 01:30:54.544860 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-e58fb533-4cd7-4113-942a-a329b041b560 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:54.545685 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-e58fb533-4cd7-4113-942a-a329b041b560 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:54.547829 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1411/2837] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 01:30:54 2022] DELETE /identity/v3/projects/ee99d9b8ed0f4e9297aff9e64fa53ff7 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:30:54.643277 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdaaf27b-6a4b-4ebb-99dd-753a784f8ae2 None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:30:54.655381 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdaaf27b-6a4b-4ebb-99dd-753a784f8ae2 None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:30:54.722360 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7bbb16-d3ca-4d23-8602-f586abacabcd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0817c3320d5d41d8bba0550c8ae8dd60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:30:54.722998 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7bbb16-d3ca-4d23-8602-f586abacabcd service neutron] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:54.723293 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7bbb16-d3ca-4d23-8602-f586abacabcd service neutron] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:54.724195 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7bbb16-d3ca-4d23-8602-f586abacabcd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:54.726093 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-4a7bbb16-d3ca-4d23-8602-f586abacabcd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:30:54.737255 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7bbb16-d3ca-4d23-8602-f586abacabcd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:30:54.738757 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7bbb16-d3ca-4d23-8602-f586abacabcd service neutron] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:30:54.749645 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1427/2838] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:30:54 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:30:55.158656 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e216a540-9eac-4145-a478-ee59519ddce9 None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:55.158929 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e216a540-9eac-4145-a478-ee59519ddce9 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:55.159210 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e216a540-9eac-4145-a478-ee59519ddce9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:55.194475 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-e216a540-9eac-4145-a478-ee59519ddce9 None None] MFA Rules not processed for user `f3e944928454456cabf55a6871dd9cf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:30:55.195959 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-e216a540-9eac-4145-a478-ee59519ddce9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:30:55.297377 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1412/2839] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:30:55 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:30:55.308994 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7bbb16-d3ca-4d23-8602-f586abacabcd service neutron] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:30:55.310785 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-4a7bbb16-d3ca-4d23-8602-f586abacabcd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:30:55.383977 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7bbb16-d3ca-4d23-8602-f586abacabcd service neutron] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:30:55.393417 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8072465-3fe0-4e85-9c3a-123125ee01d1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5c8fc69c518412ba1aad61a9f4f558c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:30:55.394237 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8072465-3fe0-4e85-9c3a-123125ee01d1 service cinder] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:55.394564 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8072465-3fe0-4e85-9c3a-123125ee01d1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:55.394847 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8072465-3fe0-4e85-9c3a-123125ee01d1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:55.422921 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8072465-3fe0-4e85-9c3a-123125ee01d1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:30:55.424668 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8072465-3fe0-4e85-9c3a-123125ee01d1 service cinder] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:30:55.436310 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1428/2840] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:30:55 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:30:57.118802 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd35559e-a1f4-4be0-9dc8-3f1a2a4a720f None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:57.119752 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd35559e-a1f4-4be0-9dc8-3f1a2a4a720f None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:57.120206 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd35559e-a1f4-4be0-9dc8-3f1a2a4a720f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:57.159257 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-dd35559e-a1f4-4be0-9dc8-3f1a2a4a720f None None] MFA Rules not processed for user `f3e944928454456cabf55a6871dd9cf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:30:57.161519 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-dd35559e-a1f4-4be0-9dc8-3f1a2a4a720f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:30:57.172309 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1413/2841] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:30:57 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:30:58.074905 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8072465-3fe0-4e85-9c3a-123125ee01d1 service cinder] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:30:58.077003 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-d8072465-3fe0-4e85-9c3a-123125ee01d1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:30:58.163168 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8072465-3fe0-4e85-9c3a-123125ee01d1 service cinder] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:30:58.168591 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348fd9db-d7c0-474e-b286-84461f5c1537 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:30:58.169810 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348fd9db-d7c0-474e-b286-84461f5c1537 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:58.170114 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348fd9db-d7c0-474e-b286-84461f5c1537 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:58.170389 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348fd9db-d7c0-474e-b286-84461f5c1537 admin admin] PATH_INFO: `/v3/users/cb0154f4a3704bd0b746071bdfb90420` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:58.171521 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348fd9db-d7c0-474e-b286-84461f5c1537 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cb0154f4a3704bd0b746071bdfb90420)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:30:58.173559 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348fd9db-d7c0-474e-b286-84461f5c1537 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:30:58.188044 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:30:58.366668 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-348fd9db-d7c0-474e-b286-84461f5c1537 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cb0154f4a3704bd0b746071bdfb90420'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:58.379923 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-348fd9db-d7c0-474e-b286-84461f5c1537 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cb0154f4a3704bd0b746071bdfb90420'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:58.380645 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-348fd9db-d7c0-474e-b286-84461f5c1537 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cb0154f4a3704bd0b746071bdfb90420'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:58.387934 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-348fd9db-d7c0-474e-b286-84461f5c1537 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cb0154f4a3704bd0b746071bdfb90420'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:58.437653 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-348fd9db-d7c0-474e-b286-84461f5c1537 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cb0154f4a3704bd0b746071bdfb90420'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:58.443936 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1429/2842] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 01:30:58 2022] DELETE /identity/v3/users/cb0154f4a3704bd0b746071bdfb90420 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:30:58.725578 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd35559e-a1f4-4be0-9dc8-3f1a2a4a720f None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:30:58.737708 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd35559e-a1f4-4be0-9dc8-3f1a2a4a720f None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:30:58.812013 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03a22b0-1450-4526-abb2-20c7a57c13f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:30:58.812683 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03a22b0-1450-4526-abb2-20c7a57c13f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:58.812991 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03a22b0-1450-4526-abb2-20c7a57c13f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:58.813278 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03a22b0-1450-4526-abb2-20c7a57c13f9 admin admin] PATH_INFO: `/v3/projects/9befc65a9da84a99942b803ac6a15a0b` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:58.814435 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03a22b0-1450-4526-abb2-20c7a57c13f9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9befc65a9da84a99942b803ac6a15a0b)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:30:58.816390 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03a22b0-1450-4526-abb2-20c7a57c13f9 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:30:58.825568 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:30:58.912283 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-c03a22b0-1450-4526-abb2-20c7a57c13f9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9befc65a9da84a99942b803ac6a15a0b', 'request_id': 'req-c03a22b0-1450-4526-abb2-20c7a57c13f9'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:58.918629 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-c03a22b0-1450-4526-abb2-20c7a57c13f9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9befc65a9da84a99942b803ac6a15a0b', 'request_id': 'req-c03a22b0-1450-4526-abb2-20c7a57c13f9'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:58.926052 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-c03a22b0-1450-4526-abb2-20c7a57c13f9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9befc65a9da84a99942b803ac6a15a0b', 'request_id': 'req-c03a22b0-1450-4526-abb2-20c7a57c13f9'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:58.997136 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-c03a22b0-1450-4526-abb2-20c7a57c13f9 admin admin] The token cache is being invalidate because project 9befc65a9da84a99942b803ac6a15a0b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81749) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 01:30:58.997467 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-c03a22b0-1450-4526-abb2-20c7a57c13f9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:58.998088 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-c03a22b0-1450-4526-abb2-20c7a57c13f9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:58.999901 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1414/2843] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 01:30:58 2022] DELETE /identity/v3/projects/9befc65a9da84a99942b803ac6a15a0b => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:30:59.003883 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348fd9db-d7c0-474e-b286-84461f5c1537 admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:30:59.005461 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-348fd9db-d7c0-474e-b286-84461f5c1537 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:30:59.019408 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348fd9db-d7c0-474e-b286-84461f5c1537 admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:30:59.077411 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83ab631-8642-4096-94ec-18c80362f052 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:30:59.078057 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83ab631-8642-4096-94ec-18c80362f052 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:59.078316 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83ab631-8642-4096-94ec-18c80362f052 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:59.078594 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83ab631-8642-4096-94ec-18c80362f052 admin admin] PATH_INFO: `/v3/users/340bc7c2d6d44ccb84e04909e0ff1514` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:59.079711 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83ab631-8642-4096-94ec-18c80362f052 admin admin] RBAC: Authorizing `identity:delete_user(user_id=340bc7c2d6d44ccb84e04909e0ff1514)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:30:59.081958 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83ab631-8642-4096-94ec-18c80362f052 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:30:59.238930 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-d83ab631-8642-4096-94ec-18c80362f052 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '340bc7c2d6d44ccb84e04909e0ff1514'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:59.253558 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-d83ab631-8642-4096-94ec-18c80362f052 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '340bc7c2d6d44ccb84e04909e0ff1514'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:59.254339 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-d83ab631-8642-4096-94ec-18c80362f052 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '340bc7c2d6d44ccb84e04909e0ff1514'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:59.263472 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-d83ab631-8642-4096-94ec-18c80362f052 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '340bc7c2d6d44ccb84e04909e0ff1514'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:59.340476 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-d83ab631-8642-4096-94ec-18c80362f052 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '340bc7c2d6d44ccb84e04909e0ff1514'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:59.347276 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1430/2844] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 01:30:59 2022] DELETE /identity/v3/users/340bc7c2d6d44ccb84e04909e0ff1514 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:30:59.586790 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03a22b0-1450-4526-abb2-20c7a57c13f9 admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:30:59.586790 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-c03a22b0-1450-4526-abb2-20c7a57c13f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:30:59.606328 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03a22b0-1450-4526-abb2-20c7a57c13f9 admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:30:59.670628 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5003f29-8c06-42b8-9dd2-b6837cd90603 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:30:59.671312 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5003f29-8c06-42b8-9dd2-b6837cd90603 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:30:59.671598 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5003f29-8c06-42b8-9dd2-b6837cd90603 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:30:59.671879 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5003f29-8c06-42b8-9dd2-b6837cd90603 admin admin] PATH_INFO: `/v3/projects/5987e5c9567141e09f55b0e723d2133d` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:30:59.673084 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5003f29-8c06-42b8-9dd2-b6837cd90603 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5987e5c9567141e09f55b0e723d2133d)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:30:59.675240 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5003f29-8c06-42b8-9dd2-b6837cd90603 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:30:59.776073 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-a5003f29-8c06-42b8-9dd2-b6837cd90603 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5987e5c9567141e09f55b0e723d2133d', 'request_id': 'req-a5003f29-8c06-42b8-9dd2-b6837cd90603'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:59.781805 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-a5003f29-8c06-42b8-9dd2-b6837cd90603 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5987e5c9567141e09f55b0e723d2133d', 'request_id': 'req-a5003f29-8c06-42b8-9dd2-b6837cd90603'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:59.788749 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-a5003f29-8c06-42b8-9dd2-b6837cd90603 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5987e5c9567141e09f55b0e723d2133d', 'request_id': 'req-a5003f29-8c06-42b8-9dd2-b6837cd90603'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:59.848770 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-a5003f29-8c06-42b8-9dd2-b6837cd90603 admin admin] The token cache is being invalidate because project 5987e5c9567141e09f55b0e723d2133d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81749) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 01:30:59.849045 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-a5003f29-8c06-42b8-9dd2-b6837cd90603 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:59.849677 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-a5003f29-8c06-42b8-9dd2-b6837cd90603 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:30:59.851147 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1415/2845] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 01:30:59 2022] DELETE /identity/v3/projects/5987e5c9567141e09f55b0e723d2133d => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:31:09.343447 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=74,keystone:DELETE=19,keystone:INSERT=2 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:31:09.845235 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=72,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:31:11.913821 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc2fdaf-ded2-465c-a43d-2ed4901f985f None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:31:11.914237 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc2fdaf-ded2-465c-a43d-2ed4901f985f None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:31:11.914387 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc2fdaf-ded2-465c-a43d-2ed4901f985f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:31:11.936340 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-9fc2fdaf-ded2-465c-a43d-2ed4901f985f None None] MFA Rules not processed for user `78b0a7bb7fed432cadfdd79c2ddab8c4`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:31:11.937497 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-9fc2fdaf-ded2-465c-a43d-2ed4901f985f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:31:11.993934 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1431/2846] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:31:11 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:31:21.987140 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=25 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:31:34.134348 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d072d5af-80ed-41f3-9092-3856999507b8 None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:31:34.134836 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d072d5af-80ed-41f3-9092-3856999507b8 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:31:34.134972 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d072d5af-80ed-41f3-9092-3856999507b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:31:34.155355 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-d072d5af-80ed-41f3-9092-3856999507b8 None None] MFA Rules not processed for user `b3efb4f9daac4a0ca05a36dd361f17e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:31:34.156616 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-d072d5af-80ed-41f3-9092-3856999507b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:31:34.211620 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1416/2847] 10.222.0.20 () {58 vars in 1009 bytes} [Wed Aug 31 01:31:34 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 01:31:34.223816 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc2fdaf-ded2-465c-a43d-2ed4901f985f None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:31:34.225282 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-9fc2fdaf-ded2-465c-a43d-2ed4901f985f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:31:34.293290 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc2fdaf-ded2-465c-a43d-2ed4901f985f None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:31:34.298775 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f68e78-b03e-4aae-bee6-a71def241903 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc1d010919ec40878ae829cc8dceec4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:31:34.299386 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f68e78-b03e-4aae-bee6-a71def241903 service swift] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:31:34.299672 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f68e78-b03e-4aae-bee6-a71def241903 service swift] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:31:34.299969 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f68e78-b03e-4aae-bee6-a71def241903 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:31:34.308129 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f68e78-b03e-4aae-bee6-a71def241903 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:31:34.309710 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f68e78-b03e-4aae-bee6-a71def241903 service swift] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:31:34.318640 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1432/2848] 10.222.0.20 () {60 vars in 1384 bytes} [Wed Aug 31 01:31:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 01:31:44.204857 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=25 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:31:44.315229 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=32 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:31:50.814291 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51684673-91c3-4d61-aa33-3560831ed526 None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:31:50.814751 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51684673-91c3-4d61-aa33-3560831ed526 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:31:50.814862 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51684673-91c3-4d61-aa33-3560831ed526 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:31:50.833904 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-51684673-91c3-4d61-aa33-3560831ed526 None None] MFA Rules not processed for user `b3efb4f9daac4a0ca05a36dd361f17e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:31:50.835112 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-51684673-91c3-4d61-aa33-3560831ed526 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:31:50.842371 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1417/2849] 10.222.0.20 () {58 vars in 967 bytes} [Wed Aug 31 01:31:50 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:31:50.855465 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f68e78-b03e-4aae-bee6-a71def241903 service swift] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:31:50.866799 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f68e78-b03e-4aae-bee6-a71def241903 service swift] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:31:50.871896 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e31abc-10b5-47c5-b6b7-ebb36d3923ae service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc1d010919ec40878ae829cc8dceec4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:31:50.872484 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e31abc-10b5-47c5-b6b7-ebb36d3923ae service swift] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:31:50.872787 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e31abc-10b5-47c5-b6b7-ebb36d3923ae service swift] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:31:50.873059 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e31abc-10b5-47c5-b6b7-ebb36d3923ae service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:31:50.881183 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e31abc-10b5-47c5-b6b7-ebb36d3923ae service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:31:50.882930 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e31abc-10b5-47c5-b6b7-ebb36d3923ae service swift] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:31:50.891157 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1433/2850] 10.222.0.20 () {60 vars in 1384 bytes} [Wed Aug 31 01:31:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 01:31:55.992643 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51684673-91c3-4d61-aa33-3560831ed526 None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:31:55.994161 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-51684673-91c3-4d61-aa33-3560831ed526 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:31:56.058538 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51684673-91c3-4d61-aa33-3560831ed526 None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:31:56.063070 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e892792-f473-4bed-bcd7-4d6a1bdab775 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:31:56.063718 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e892792-f473-4bed-bcd7-4d6a1bdab775 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:31:56.064005 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e892792-f473-4bed-bcd7-4d6a1bdab775 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:31:56.064285 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e892792-f473-4bed-bcd7-4d6a1bdab775 admin admin] PATH_INFO: `/v3/users/78b0a7bb7fed432cadfdd79c2ddab8c4` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:31:56.065470 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e892792-f473-4bed-bcd7-4d6a1bdab775 admin admin] RBAC: Authorizing `identity:delete_user(user_id=78b0a7bb7fed432cadfdd79c2ddab8c4)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:31:56.067397 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e892792-f473-4bed-bcd7-4d6a1bdab775 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:31:56.255671 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-8e892792-f473-4bed-bcd7-4d6a1bdab775 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '78b0a7bb7fed432cadfdd79c2ddab8c4'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:31:56.333203 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-8e892792-f473-4bed-bcd7-4d6a1bdab775 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '78b0a7bb7fed432cadfdd79c2ddab8c4'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:31:56.333909 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-8e892792-f473-4bed-bcd7-4d6a1bdab775 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '78b0a7bb7fed432cadfdd79c2ddab8c4'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:31:56.341757 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-8e892792-f473-4bed-bcd7-4d6a1bdab775 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '78b0a7bb7fed432cadfdd79c2ddab8c4'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:31:56.342522 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-8e892792-f473-4bed-bcd7-4d6a1bdab775 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '78b0a7bb7fed432cadfdd79c2ddab8c4'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:31:56.366213 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1418/2851] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 01:31:55 2022] DELETE /identity/v3/users/78b0a7bb7fed432cadfdd79c2ddab8c4 => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:31:56.632449 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e31abc-10b5-47c5-b6b7-ebb36d3923ae service swift] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:31:56.645018 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e31abc-10b5-47c5-b6b7-ebb36d3923ae service swift] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:31:56.703817 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8347706a-5eb8-4dad-8f95-9f802d7e7058 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:31:56.705151 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8347706a-5eb8-4dad-8f95-9f802d7e7058 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:31:56.705684 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8347706a-5eb8-4dad-8f95-9f802d7e7058 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:31:56.705959 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8347706a-5eb8-4dad-8f95-9f802d7e7058 admin admin] PATH_INFO: `/v3/projects/f9084d8c977147b7a3652e34466b2522` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:31:56.707070 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8347706a-5eb8-4dad-8f95-9f802d7e7058 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f9084d8c977147b7a3652e34466b2522)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:31:56.708995 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8347706a-5eb8-4dad-8f95-9f802d7e7058 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:31:56.807182 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-8347706a-5eb8-4dad-8f95-9f802d7e7058 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f9084d8c977147b7a3652e34466b2522', 'request_id': 'req-8347706a-5eb8-4dad-8f95-9f802d7e7058'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:31:56.812477 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-8347706a-5eb8-4dad-8f95-9f802d7e7058 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f9084d8c977147b7a3652e34466b2522', 'request_id': 'req-8347706a-5eb8-4dad-8f95-9f802d7e7058'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:31:56.864899 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-8347706a-5eb8-4dad-8f95-9f802d7e7058 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f9084d8c977147b7a3652e34466b2522', 'request_id': 'req-8347706a-5eb8-4dad-8f95-9f802d7e7058'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:31:56.875992 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-8347706a-5eb8-4dad-8f95-9f802d7e7058 admin admin] The token cache is being invalidate because project f9084d8c977147b7a3652e34466b2522 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81750) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 01:31:56.876877 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-8347706a-5eb8-4dad-8f95-9f802d7e7058 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:31:56.877511 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-8347706a-5eb8-4dad-8f95-9f802d7e7058 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:31:56.878992 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1434/2852] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 01:31:56 2022] DELETE /identity/v3/projects/f9084d8c977147b7a3652e34466b2522 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:31:56.883888 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef1e646-b1d9-4950-9c9f-feb7b9c269f3 None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:31:56.884196 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef1e646-b1d9-4950-9c9f-feb7b9c269f3 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:31:56.884477 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef1e646-b1d9-4950-9c9f-feb7b9c269f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:31:56.904749 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-fef1e646-b1d9-4950-9c9f-feb7b9c269f3 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:31:56.905956 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-fef1e646-b1d9-4950-9c9f-feb7b9c269f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:31:56.964535 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1419/2853] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:31:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:31:56.974977 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d457f3-d360-4a06-8626-091cbb21f887 None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:31:56.975267 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d457f3-d360-4a06-8626-091cbb21f887 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:31:56.975547 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d457f3-d360-4a06-8626-091cbb21f887 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:31:56.995421 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-a1d457f3-d360-4a06-8626-091cbb21f887 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:31:56.996632 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-a1d457f3-d360-4a06-8626-091cbb21f887 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:31:57.005586 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1435/2854] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:31:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:31:57.009674 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef1e646-b1d9-4950-9c9f-feb7b9c269f3 None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:31:57.019440 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef1e646-b1d9-4950-9c9f-feb7b9c269f3 None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:31:57.024093 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be75eeb-d4df-44ac-a685-6b7062932917 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:31:57.024688 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be75eeb-d4df-44ac-a685-6b7062932917 admin admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:31:57.025017 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be75eeb-d4df-44ac-a685-6b7062932917 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:31:57.025279 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be75eeb-d4df-44ac-a685-6b7062932917 admin admin] PATH_INFO: `/v3/domains` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:31:57.025925 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be75eeb-d4df-44ac-a685-6b7062932917 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:31:57.026232 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be75eeb-d4df-44ac-a685-6b7062932917 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:31:57.027827 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be75eeb-d4df-44ac-a685-6b7062932917 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:31:57.036293 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:31:57.039476 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1420/2855] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Aug 31 01:31:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:31:57.043868 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d457f3-d360-4a06-8626-091cbb21f887 None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:31:57.053631 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d457f3-d360-4a06-8626-091cbb21f887 None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:31:57.058340 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43a1e02-b7d8-46f6-9d21-48f43281f28b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:31:57.058877 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43a1e02-b7d8-46f6-9d21-48f43281f28b admin admin] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:31:57.059158 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43a1e02-b7d8-46f6-9d21-48f43281f28b admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:31:57.059431 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43a1e02-b7d8-46f6-9d21-48f43281f28b admin admin] PATH_INFO: `/v3/projects` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:31:57.060059 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43a1e02-b7d8-46f6-9d21-48f43281f28b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:31:57.061985 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43a1e02-b7d8-46f6-9d21-48f43281f28b admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:31:57.122563 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1436/2856] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 01:31:57 2022] POST /identity/v3/projects => generated 385 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:31:57.127320 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be75eeb-d4df-44ac-a685-6b7062932917 admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:31:57.137486 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be75eeb-d4df-44ac-a685-6b7062932917 admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:31:57.192261 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad89ca89-2f58-400a-82e4-bad6796f640c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:31:57.192896 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad89ca89-2f58-400a-82e4-bad6796f640c admin admin] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:31:57.193181 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad89ca89-2f58-400a-82e4-bad6796f640c admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:31:57.193458 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad89ca89-2f58-400a-82e4-bad6796f640c admin admin] PATH_INFO: `/v3/users` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:31:57.194085 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad89ca89-2f58-400a-82e4-bad6796f640c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:31:57.195987 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad89ca89-2f58-400a-82e4-bad6796f640c admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:31:57.281111 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1421/2857] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 01:31:57 2022] POST /identity/v3/users => generated 293 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:31:57.285874 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43a1e02-b7d8-46f6-9d21-48f43281f28b admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:31:57.296155 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43a1e02-b7d8-46f6-9d21-48f43281f28b admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:31:57.301360 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd953a4-d494-47e3-b711-e53b2201be0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:31:57.301948 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd953a4-d494-47e3-b711-e53b2201be0d admin admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:31:57.302225 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd953a4-d494-47e3-b711-e53b2201be0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:31:57.302498 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd953a4-d494-47e3-b711-e53b2201be0d admin admin] PATH_INFO: `/v3/roles` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:31:57.303188 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd953a4-d494-47e3-b711-e53b2201be0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:31:57.304783 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd953a4-d494-47e3-b711-e53b2201be0d admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:31:57.314431 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1437/2858] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:31:57 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:31:57.318121 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad89ca89-2f58-400a-82e4-bad6796f640c admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:31:57.328053 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad89ca89-2f58-400a-82e4-bad6796f640c admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:31:57.332975 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2b6a22-7973-45e7-b3f7-88d3f9d5e206 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:31:57.333571 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2b6a22-7973-45e7-b3f7-88d3f9d5e206 admin admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:31:57.333858 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2b6a22-7973-45e7-b3f7-88d3f9d5e206 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:31:57.334136 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2b6a22-7973-45e7-b3f7-88d3f9d5e206 admin admin] PATH_INFO: `/v3/roles` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:31:57.334773 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2b6a22-7973-45e7-b3f7-88d3f9d5e206 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:31:57.336359 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2b6a22-7973-45e7-b3f7-88d3f9d5e206 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:31:57.347202 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1422/2859] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:31:57 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:31:57.351148 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd953a4-d494-47e3-b711-e53b2201be0d admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:31:57.361782 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd953a4-d494-47e3-b711-e53b2201be0d admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:31:57.366835 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63075aa5-79f3-46f7-b677-127bc2139499 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:31:57.367494 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63075aa5-79f3-46f7-b677-127bc2139499 admin admin] REQUEST_METHOD: `PUT` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:31:57.367778 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63075aa5-79f3-46f7-b677-127bc2139499 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:31:57.368060 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63075aa5-79f3-46f7-b677-127bc2139499 admin admin] PATH_INFO: `/v3/projects/f5d1d9075352420e808d355fd100f44a/users/79c261247bb54dca934fcacdf65cbc01/roles/d0b9768f4d014766b9326387f2e11f34` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:31:57.383335 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63075aa5-79f3-46f7-b677-127bc2139499 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f5d1d9075352420e808d355fd100f44a, user_id=79c261247bb54dca934fcacdf65cbc01, role_id=d0b9768f4d014766b9326387f2e11f34)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:31:57.386024 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63075aa5-79f3-46f7-b677-127bc2139499 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:31:57.447624 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1438/2860] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 01:31:57 2022] PUT /identity/v3/projects/f5d1d9075352420e808d355fd100f44a/users/79c261247bb54dca934fcacdf65cbc01/roles/d0b9768f4d014766b9326387f2e11f34 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:31:57.458320 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8872922e-6412-455a-9db3-7088b71da5e8 None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:31:57.458692 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8872922e-6412-455a-9db3-7088b71da5e8 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:31:57.459001 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8872922e-6412-455a-9db3-7088b71da5e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:31:57.490222 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-8872922e-6412-455a-9db3-7088b71da5e8 None None] MFA Rules not processed for user `79c261247bb54dca934fcacdf65cbc01`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:31:57.491451 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-8872922e-6412-455a-9db3-7088b71da5e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:31:57.557200 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1423/2861] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:31:57 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:31:57.569409 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63075aa5-79f3-46f7-b677-127bc2139499 admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:31:57.570663 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-63075aa5-79f3-46f7-b677-127bc2139499 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:31:57.585344 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:31:57.628610 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63075aa5-79f3-46f7-b677-127bc2139499 admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:31:57.632350 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792f0bde-b844-4fd1-a474-486919295365 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5c8fc69c518412ba1aad61a9f4f558c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:31:57.632991 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792f0bde-b844-4fd1-a474-486919295365 service cinder] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:31:57.633547 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792f0bde-b844-4fd1-a474-486919295365 service cinder] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:31:57.633547 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792f0bde-b844-4fd1-a474-486919295365 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:31:57.642345 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792f0bde-b844-4fd1-a474-486919295365 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:31:57.643793 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792f0bde-b844-4fd1-a474-486919295365 service cinder] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:31:57.652545 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1439/2862] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:31:57 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:32:00.664694 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8872922e-6412-455a-9db3-7088b71da5e8 None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:32:00.725774 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8872922e-6412-455a-9db3-7088b71da5e8 None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:32:00.730246 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ad2911-a330-4045-b2d1-57e0c2782aa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:32:00.730866 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ad2911-a330-4045-b2d1-57e0c2782aa9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:32:00.731151 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ad2911-a330-4045-b2d1-57e0c2782aa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:32:00.731433 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ad2911-a330-4045-b2d1-57e0c2782aa9 admin admin] PATH_INFO: `/v3/users/79c261247bb54dca934fcacdf65cbc01` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:32:00.732553 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ad2911-a330-4045-b2d1-57e0c2782aa9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=79c261247bb54dca934fcacdf65cbc01)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:32:00.734505 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ad2911-a330-4045-b2d1-57e0c2782aa9 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:32:00.745049 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:32:00.917557 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-06ad2911-a330-4045-b2d1-57e0c2782aa9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '79c261247bb54dca934fcacdf65cbc01'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:32:00.995962 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-06ad2911-a330-4045-b2d1-57e0c2782aa9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '79c261247bb54dca934fcacdf65cbc01'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:32:00.996638 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-06ad2911-a330-4045-b2d1-57e0c2782aa9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '79c261247bb54dca934fcacdf65cbc01'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:32:01.005177 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-06ad2911-a330-4045-b2d1-57e0c2782aa9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '79c261247bb54dca934fcacdf65cbc01'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:32:01.008624 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-06ad2911-a330-4045-b2d1-57e0c2782aa9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '79c261247bb54dca934fcacdf65cbc01'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:32:01.023446 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1424/2863] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 01:32:00 2022] DELETE /identity/v3/users/79c261247bb54dca934fcacdf65cbc01 => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:32:01.028070 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab2887e-1424-473b-ab14-38d8a1a9daa3 None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:32:01.028527 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab2887e-1424-473b-ab14-38d8a1a9daa3 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:32:01.028643 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab2887e-1424-473b-ab14-38d8a1a9daa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:32:01.048674 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-7ab2887e-1424-473b-ab14-38d8a1a9daa3 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:32:01.050088 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-7ab2887e-1424-473b-ab14-38d8a1a9daa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:32:01.107778 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1440/2864] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:32:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:32:01.117540 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ad2911-a330-4045-b2d1-57e0c2782aa9 admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:32:01.118924 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-06ad2911-a330-4045-b2d1-57e0c2782aa9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:32:01.175426 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ad2911-a330-4045-b2d1-57e0c2782aa9 admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:32:01.179000 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d72ed7-7ca8-4af7-be99-522ae79940c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0817c3320d5d41d8bba0550c8ae8dd60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:32:01.179598 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d72ed7-7ca8-4af7-be99-522ae79940c1 service neutron] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:32:01.179883 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d72ed7-7ca8-4af7-be99-522ae79940c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:32:01.180163 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d72ed7-7ca8-4af7-be99-522ae79940c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:32:01.189488 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d72ed7-7ca8-4af7-be99-522ae79940c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:32:01.190922 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d72ed7-7ca8-4af7-be99-522ae79940c1 service neutron] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:32:01.200355 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1425/2865] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:32:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:32:01.589697 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab2887e-1424-473b-ab14-38d8a1a9daa3 None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:32:01.591271 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-7ab2887e-1424-473b-ab14-38d8a1a9daa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:32:01.603531 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab2887e-1424-473b-ab14-38d8a1a9daa3 None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:32:01.608226 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e48da6-c8b0-4848-a76f-03949c003147 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:32:01.609084 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e48da6-c8b0-4848-a76f-03949c003147 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:32:01.609374 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e48da6-c8b0-4848-a76f-03949c003147 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:32:01.609647 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e48da6-c8b0-4848-a76f-03949c003147 admin admin] PATH_INFO: `/v3/projects/f5d1d9075352420e808d355fd100f44a` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:32:01.611933 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e48da6-c8b0-4848-a76f-03949c003147 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f5d1d9075352420e808d355fd100f44a)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:32:01.613860 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e48da6-c8b0-4848-a76f-03949c003147 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:32:01.699634 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-59e48da6-c8b0-4848-a76f-03949c003147 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f5d1d9075352420e808d355fd100f44a', 'request_id': 'req-59e48da6-c8b0-4848-a76f-03949c003147'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:32:01.705124 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-59e48da6-c8b0-4848-a76f-03949c003147 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f5d1d9075352420e808d355fd100f44a', 'request_id': 'req-59e48da6-c8b0-4848-a76f-03949c003147'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:32:01.772424 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-59e48da6-c8b0-4848-a76f-03949c003147 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f5d1d9075352420e808d355fd100f44a', 'request_id': 'req-59e48da6-c8b0-4848-a76f-03949c003147'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:32:01.783916 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-59e48da6-c8b0-4848-a76f-03949c003147 admin admin] The token cache is being invalidate because project f5d1d9075352420e808d355fd100f44a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81750) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 01:32:01.784212 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-59e48da6-c8b0-4848-a76f-03949c003147 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:32:01.784871 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-59e48da6-c8b0-4848-a76f-03949c003147 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:32:01.786329 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1441/2866] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 01:32:01 2022] DELETE /identity/v3/projects/f5d1d9075352420e808d355fd100f44a => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:32:01.791517 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49510d4-81f5-4975-bb85-7cb057785735 None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:32:01.792024 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49510d4-81f5-4975-bb85-7cb057785735 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:32:01.792140 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49510d4-81f5-4975-bb85-7cb057785735 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:32:01.811609 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-f49510d4-81f5-4975-bb85-7cb057785735 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:32:01.812839 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-f49510d4-81f5-4975-bb85-7cb057785735 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:32:01.872220 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1426/2867] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:32:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:32:01.880095 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5916fa62-b564-46e3-95bd-49b7f25d780b None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:32:01.880484 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5916fa62-b564-46e3-95bd-49b7f25d780b None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:32:01.880794 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5916fa62-b564-46e3-95bd-49b7f25d780b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:32:01.901516 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-5916fa62-b564-46e3-95bd-49b7f25d780b None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:32:01.902901 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-5916fa62-b564-46e3-95bd-49b7f25d780b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:32:01.912835 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1442/2868] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:32:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 01:32:01.917326 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49510d4-81f5-4975-bb85-7cb057785735 None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:32:01.927809 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49510d4-81f5-4975-bb85-7cb057785735 None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:32:01.932629 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02785b0f-81a8-4622-b73f-41041b50e10b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:32:01.933265 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02785b0f-81a8-4622-b73f-41041b50e10b admin admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:32:01.933549 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02785b0f-81a8-4622-b73f-41041b50e10b admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:32:01.933826 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02785b0f-81a8-4622-b73f-41041b50e10b admin admin] PATH_INFO: `/v3/domains` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:32:01.934472 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02785b0f-81a8-4622-b73f-41041b50e10b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:32:01.934777 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02785b0f-81a8-4622-b73f-41041b50e10b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:32:01.936378 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02785b0f-81a8-4622-b73f-41041b50e10b admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:32:01.948307 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1427/2869] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Aug 31 01:32:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:32:01.952765 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5916fa62-b564-46e3-95bd-49b7f25d780b None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:32:01.962300 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5916fa62-b564-46e3-95bd-49b7f25d780b None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:32:01.967116 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2060ba9d-27f3-4a60-acfd-62d28ff64044 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:32:01.967717 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2060ba9d-27f3-4a60-acfd-62d28ff64044 admin admin] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:32:01.967989 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2060ba9d-27f3-4a60-acfd-62d28ff64044 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:32:01.968264 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2060ba9d-27f3-4a60-acfd-62d28ff64044 admin admin] PATH_INFO: `/v3/projects` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:32:01.968922 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2060ba9d-27f3-4a60-acfd-62d28ff64044 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:32:01.971310 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2060ba9d-27f3-4a60-acfd-62d28ff64044 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:32:02.061659 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1443/2870] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 01:32:01 2022] POST /identity/v3/projects => generated 371 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:32:02.065729 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02785b0f-81a8-4622-b73f-41041b50e10b admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:32:02.076457 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02785b0f-81a8-4622-b73f-41041b50e10b admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:32:02.089713 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:32:02.135348 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0723210a-0dbc-4e04-bcad-a38d19e022ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:32:02.136071 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0723210a-0dbc-4e04-bcad-a38d19e022ef admin admin] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:32:02.136390 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0723210a-0dbc-4e04-bcad-a38d19e022ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:32:02.136666 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0723210a-0dbc-4e04-bcad-a38d19e022ef admin admin] PATH_INFO: `/v3/users` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:32:02.137338 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0723210a-0dbc-4e04-bcad-a38d19e022ef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:32:02.139329 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0723210a-0dbc-4e04-bcad-a38d19e022ef admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:32:02.225611 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1428/2871] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 01:32:02 2022] POST /identity/v3/users => generated 286 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:32:02.229715 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2060ba9d-27f3-4a60-acfd-62d28ff64044 admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:32:02.240310 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2060ba9d-27f3-4a60-acfd-62d28ff64044 admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:32:02.244974 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86125f6f-2399-43cf-925c-6c5ecad8b071 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:32:02.245566 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86125f6f-2399-43cf-925c-6c5ecad8b071 admin admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:32:02.245845 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86125f6f-2399-43cf-925c-6c5ecad8b071 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:32:02.246138 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86125f6f-2399-43cf-925c-6c5ecad8b071 admin admin] PATH_INFO: `/v3/roles` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:32:02.246798 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86125f6f-2399-43cf-925c-6c5ecad8b071 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:32:02.248361 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86125f6f-2399-43cf-925c-6c5ecad8b071 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:32:02.253575 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:32:02.258336 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1444/2872] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:32:02 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:32:02.262231 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0723210a-0dbc-4e04-bcad-a38d19e022ef admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:32:02.272700 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0723210a-0dbc-4e04-bcad-a38d19e022ef admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:32:02.277458 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190e4a0a-20b0-43af-8259-9fb0bfd0b4d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:32:02.278066 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190e4a0a-20b0-43af-8259-9fb0bfd0b4d8 admin admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:32:02.278356 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190e4a0a-20b0-43af-8259-9fb0bfd0b4d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:32:02.278657 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190e4a0a-20b0-43af-8259-9fb0bfd0b4d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:32:02.279333 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190e4a0a-20b0-43af-8259-9fb0bfd0b4d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:32:02.281125 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190e4a0a-20b0-43af-8259-9fb0bfd0b4d8 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:32:02.296006 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1429/2873] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:32:02 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:32:02.299966 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86125f6f-2399-43cf-925c-6c5ecad8b071 admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:32:02.317320 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86125f6f-2399-43cf-925c-6c5ecad8b071 admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:32:02.322968 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497a2f60-c27d-4461-9313-fc25b204763c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:32:02.323688 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497a2f60-c27d-4461-9313-fc25b204763c admin admin] REQUEST_METHOD: `PUT` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:32:02.323982 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497a2f60-c27d-4461-9313-fc25b204763c admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:32:02.328587 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497a2f60-c27d-4461-9313-fc25b204763c admin admin] PATH_INFO: `/v3/projects/970d06f2d1e24f0e9072a7f43bb4925b/users/181421ac2d7b49d496cabfd6f205f203/roles/d0b9768f4d014766b9326387f2e11f34` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:32:02.340214 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497a2f60-c27d-4461-9313-fc25b204763c admin admin] RBAC: Authorizing `identity:create_grant(project_id=970d06f2d1e24f0e9072a7f43bb4925b, user_id=181421ac2d7b49d496cabfd6f205f203, role_id=d0b9768f4d014766b9326387f2e11f34)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:32:02.342875 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497a2f60-c27d-4461-9313-fc25b204763c admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:32:02.397565 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1445/2874] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 01:32:02 2022] PUT /identity/v3/projects/970d06f2d1e24f0e9072a7f43bb4925b/users/181421ac2d7b49d496cabfd6f205f203/roles/d0b9768f4d014766b9326387f2e11f34 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 01:32:02.403852 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f63e29-f822-4524-a224-efad6dd5c468 None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:32:02.403852 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f63e29-f822-4524-a224-efad6dd5c468 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:32:02.403852 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f63e29-f822-4524-a224-efad6dd5c468 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:32:02.424149 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-f9f63e29-f822-4524-a224-efad6dd5c468 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:32:02.426683 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-f9f63e29-f822-4524-a224-efad6dd5c468 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:32:02.483540 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1430/2875] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:32:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:32:02.493609 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497a2f60-c27d-4461-9313-fc25b204763c admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:32:02.495317 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-497a2f60-c27d-4461-9313-fc25b204763c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:32:02.553928 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497a2f60-c27d-4461-9313-fc25b204763c admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:32:02.557898 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5689b934-6a29-4f00-b2df-1b6d9b216ea5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0817c3320d5d41d8bba0550c8ae8dd60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:32:02.558445 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5689b934-6a29-4f00-b2df-1b6d9b216ea5 service neutron] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:32:02.558724 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5689b934-6a29-4f00-b2df-1b6d9b216ea5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:32:02.558996 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5689b934-6a29-4f00-b2df-1b6d9b216ea5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:32:02.569273 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5689b934-6a29-4f00-b2df-1b6d9b216ea5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:32:02.570694 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5689b934-6a29-4f00-b2df-1b6d9b216ea5 service neutron] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:32:02.581335 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1446/2876] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:32:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:32:08.018626 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937a09ea-5953-4da7-abd4-c534605bab6f None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:32:08.019160 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937a09ea-5953-4da7-abd4-c534605bab6f None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:32:08.019160 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937a09ea-5953-4da7-abd4-c534605bab6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:32:08.050656 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-937a09ea-5953-4da7-abd4-c534605bab6f None None] MFA Rules not processed for user `181421ac2d7b49d496cabfd6f205f203`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:32:08.052104 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-937a09ea-5953-4da7-abd4-c534605bab6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:32:08.128216 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1431/2877] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:32:08 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:32:08.132658 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5689b934-6a29-4f00-b2df-1b6d9b216ea5 service neutron] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:32:08.143309 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5689b934-6a29-4f00-b2df-1b6d9b216ea5 service neutron] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:32:08.148195 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d23a65b-5814-45b7-8a40-6aa3e725f862 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:32:08.149004 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d23a65b-5814-45b7-8a40-6aa3e725f862 admin admin] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:32:08.149238 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d23a65b-5814-45b7-8a40-6aa3e725f862 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:32:08.149515 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d23a65b-5814-45b7-8a40-6aa3e725f862 admin admin] PATH_INFO: `/v3/projects` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:32:08.150280 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d23a65b-5814-45b7-8a40-6aa3e725f862 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:32:08.152185 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d23a65b-5814-45b7-8a40-6aa3e725f862 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:32:08.238180 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1447/2878] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 01:32:08 2022] POST /identity/v3/projects => generated 373 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:32:08.242569 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937a09ea-5953-4da7-abd4-c534605bab6f None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:32:08.258443 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937a09ea-5953-4da7-abd4-c534605bab6f None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:32:08.284777 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:32:08.316146 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df409965-86bc-4d56-bbe6-c16d741678dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:32:08.316770 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df409965-86bc-4d56-bbe6-c16d741678dc admin admin] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:32:08.317060 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df409965-86bc-4d56-bbe6-c16d741678dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:32:08.317343 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df409965-86bc-4d56-bbe6-c16d741678dc admin admin] PATH_INFO: `/v3/users` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:32:08.317986 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df409965-86bc-4d56-bbe6-c16d741678dc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:32:08.319974 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df409965-86bc-4d56-bbe6-c16d741678dc admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:32:08.390245 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1432/2879] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 01:32:08 2022] POST /identity/v3/users => generated 293 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:32:08.394780 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d23a65b-5814-45b7-8a40-6aa3e725f862 admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:32:08.405281 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d23a65b-5814-45b7-8a40-6aa3e725f862 admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:32:08.410656 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b4f873-be55-483c-955a-c224f942ed69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:32:08.411252 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b4f873-be55-483c-955a-c224f942ed69 admin admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:32:08.411531 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b4f873-be55-483c-955a-c224f942ed69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:32:08.411802 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b4f873-be55-483c-955a-c224f942ed69 admin admin] PATH_INFO: `/v3/roles` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:32:08.412439 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b4f873-be55-483c-955a-c224f942ed69 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:32:08.414071 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b4f873-be55-483c-955a-c224f942ed69 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:32:08.424614 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1448/2880] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:32:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:32:08.429167 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df409965-86bc-4d56-bbe6-c16d741678dc admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:32:08.439548 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df409965-86bc-4d56-bbe6-c16d741678dc admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:32:08.444933 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a755a7-085f-4e59-bf20-cd7f91761e17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:32:08.445613 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a755a7-085f-4e59-bf20-cd7f91761e17 admin admin] REQUEST_METHOD: `PUT` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:32:08.446026 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a755a7-085f-4e59-bf20-cd7f91761e17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:32:08.446317 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a755a7-085f-4e59-bf20-cd7f91761e17 admin admin] PATH_INFO: `/v3/projects/dd9592e823ce4809b652aa7fd2029f49/users/b6e35abdbb1f459492faf571236e1962/roles/d009411496b8452b88e77ffb8db015a7` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:32:08.462752 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a755a7-085f-4e59-bf20-cd7f91761e17 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd9592e823ce4809b652aa7fd2029f49, user_id=b6e35abdbb1f459492faf571236e1962, role_id=d009411496b8452b88e77ffb8db015a7)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:32:08.465508 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a755a7-085f-4e59-bf20-cd7f91761e17 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:32:08.533149 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1433/2881] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 01:32:08 2022] PUT /identity/v3/projects/dd9592e823ce4809b652aa7fd2029f49/users/b6e35abdbb1f459492faf571236e1962/roles/d009411496b8452b88e77ffb8db015a7 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:32:08.537238 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b4f873-be55-483c-955a-c224f942ed69 admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:32:08.547127 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b4f873-be55-483c-955a-c224f942ed69 admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:32:08.604533 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1d03c8-ebcf-4bbf-ac37-3db15281eb1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:32:08.605174 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1d03c8-ebcf-4bbf-ac37-3db15281eb1e admin admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:32:08.605462 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1d03c8-ebcf-4bbf-ac37-3db15281eb1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:32:08.605804 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1d03c8-ebcf-4bbf-ac37-3db15281eb1e admin admin] PATH_INFO: `/v3/roles` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:32:08.606440 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1d03c8-ebcf-4bbf-ac37-3db15281eb1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:32:08.608039 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1d03c8-ebcf-4bbf-ac37-3db15281eb1e admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:32:08.618075 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1449/2882] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:32:08 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:32:08.621900 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a755a7-085f-4e59-bf20-cd7f91761e17 admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:32:08.632493 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a755a7-085f-4e59-bf20-cd7f91761e17 admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:32:08.637003 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8faaa27-0641-4aef-8cc6-76527aaf3f1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:32:08.637640 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8faaa27-0641-4aef-8cc6-76527aaf3f1a admin admin] REQUEST_METHOD: `PUT` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:32:08.637919 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8faaa27-0641-4aef-8cc6-76527aaf3f1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:32:08.638190 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8faaa27-0641-4aef-8cc6-76527aaf3f1a admin admin] PATH_INFO: `/v3/projects/dd9592e823ce4809b652aa7fd2029f49/users/b6e35abdbb1f459492faf571236e1962/roles/d009411496b8452b88e77ffb8db015a7` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:32:08.640050 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8faaa27-0641-4aef-8cc6-76527aaf3f1a admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd9592e823ce4809b652aa7fd2029f49, user_id=b6e35abdbb1f459492faf571236e1962, role_id=d009411496b8452b88e77ffb8db015a7)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:32:08.643350 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8faaa27-0641-4aef-8cc6-76527aaf3f1a admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:32:08.653281 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1434/2883] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 01:32:08 2022] PUT /identity/v3/projects/dd9592e823ce4809b652aa7fd2029f49/users/b6e35abdbb1f459492faf571236e1962/roles/d009411496b8452b88e77ffb8db015a7 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:32:12.276342 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a570ed-68d9-4f3d-bc7a-16454eca2d20 None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:32:12.281867 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a570ed-68d9-4f3d-bc7a-16454eca2d20 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:32:12.281867 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a570ed-68d9-4f3d-bc7a-16454eca2d20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:32:12.310932 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-c5a570ed-68d9-4f3d-bc7a-16454eca2d20 None None] MFA Rules not processed for user `f3e944928454456cabf55a6871dd9cf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:32:12.312201 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-c5a570ed-68d9-4f3d-bc7a-16454eca2d20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:32:12.383876 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:32:12.439783 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1450/2884] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:32:12 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 01:32:14.512204 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19946a0-df71-4427-841a-004e4ea628cf None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:32:14.512706 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19946a0-df71-4427-841a-004e4ea628cf None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:32:14.512965 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19946a0-df71-4427-841a-004e4ea628cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:32:14.547632 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-a19946a0-df71-4427-841a-004e4ea628cf None None] MFA Rules not processed for user `b6e35abdbb1f459492faf571236e1962`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:32:14.549680 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-a19946a0-df71-4427-841a-004e4ea628cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:32:14.642055 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1435/2885] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:32:14 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:32:14.650234 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a570ed-68d9-4f3d-bc7a-16454eca2d20 None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:32:14.651716 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-c5a570ed-68d9-4f3d-bc7a-16454eca2d20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:32:14.754482 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a570ed-68d9-4f3d-bc7a-16454eca2d20 None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:32:14.760149 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a725beb0-753f-4fad-8eb2-f9654ff16278 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8744c9de803048f583d0317d01f6a540', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:32:14.760864 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a725beb0-753f-4fad-8eb2-f9654ff16278 service nova] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:32:14.761149 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a725beb0-753f-4fad-8eb2-f9654ff16278 service nova] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:32:14.761426 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a725beb0-753f-4fad-8eb2-f9654ff16278 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:32:14.820063 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a725beb0-753f-4fad-8eb2-f9654ff16278 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:32:14.821988 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a725beb0-753f-4fad-8eb2-f9654ff16278 service nova] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:32:14.832219 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1451/2886] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:32:14 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:32:18.593617 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77955cfc-54a8-41bd-b2b3-aad45e51f466 None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:32:18.596858 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77955cfc-54a8-41bd-b2b3-aad45e51f466 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:32:18.596858 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77955cfc-54a8-41bd-b2b3-aad45e51f466 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:32:18.614576 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-77955cfc-54a8-41bd-b2b3-aad45e51f466 None None] MFA Rules not processed for user `181421ac2d7b49d496cabfd6f205f203`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:32:18.615794 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-77955cfc-54a8-41bd-b2b3-aad45e51f466 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:32:18.623338 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1436/2887] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:32:18 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:32:22.739116 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a725beb0-753f-4fad-8eb2-f9654ff16278 service nova] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:32:22.799297 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a725beb0-753f-4fad-8eb2-f9654ff16278 service nova] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:32:22.802857 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a858228-1932-461c-944b-e14dc53f8cec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0817c3320d5d41d8bba0550c8ae8dd60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:32:22.803496 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a858228-1932-461c-944b-e14dc53f8cec service neutron] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:32:22.803777 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a858228-1932-461c-944b-e14dc53f8cec service neutron] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:32:22.804052 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a858228-1932-461c-944b-e14dc53f8cec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:32:22.805865 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-8a858228-1932-461c-944b-e14dc53f8cec service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:32:22.820745 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a858228-1932-461c-944b-e14dc53f8cec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:32:22.822193 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a858228-1932-461c-944b-e14dc53f8cec service neutron] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:32:22.833880 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1452/2888] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:32:22 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 01:32:24.016402 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77955cfc-54a8-41bd-b2b3-aad45e51f466 None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:32:24.026821 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77955cfc-54a8-41bd-b2b3-aad45e51f466 None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:32:24.030466 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2279f24e-5e15-4b03-a06d-53170ed5ab6a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0817c3320d5d41d8bba0550c8ae8dd60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:32:24.031074 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2279f24e-5e15-4b03-a06d-53170ed5ab6a service neutron] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:32:24.031356 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2279f24e-5e15-4b03-a06d-53170ed5ab6a service neutron] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:32:24.031634 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2279f24e-5e15-4b03-a06d-53170ed5ab6a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:32:24.033393 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-2279f24e-5e15-4b03-a06d-53170ed5ab6a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:32:24.046845 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2279f24e-5e15-4b03-a06d-53170ed5ab6a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:32:24.048820 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2279f24e-5e15-4b03-a06d-53170ed5ab6a service neutron] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:32:24.060008 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1437/2889] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:32:24 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:32:32.828383 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=97 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:32:34.054999 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=75,keystone:INSERT=5 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:32:35.250170 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a858228-1932-461c-944b-e14dc53f8cec service neutron] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:32:35.251713 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-8a858228-1932-461c-944b-e14dc53f8cec service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:32:35.322182 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a858228-1932-461c-944b-e14dc53f8cec service neutron] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:32:35.327233 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b9dd43-273b-4d4c-be2a-4b22e1f54fe3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dea0fe5b1d6b405cb47b95192580303b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:32:35.327869 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b9dd43-273b-4d4c-be2a-4b22e1f54fe3 service placement] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:32:35.328152 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b9dd43-273b-4d4c-be2a-4b22e1f54fe3 service placement] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:32:35.328430 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b9dd43-273b-4d4c-be2a-4b22e1f54fe3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:32:35.331148 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-15b9dd43-273b-4d4c-be2a-4b22e1f54fe3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:32:35.342702 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b9dd43-273b-4d4c-be2a-4b22e1f54fe3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:32:35.344315 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b9dd43-273b-4d4c-be2a-4b22e1f54fe3 service placement] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:32:35.354810 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1453/2890] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:32:35 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:32:41.074044 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2279f24e-5e15-4b03-a06d-53170ed5ab6a service neutron] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:32:41.075838 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-2279f24e-5e15-4b03-a06d-53170ed5ab6a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:32:41.088589 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2279f24e-5e15-4b03-a06d-53170ed5ab6a service neutron] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:32:41.092575 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6c4256-0339-41a8-acf0-1288df5eccca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0817c3320d5d41d8bba0550c8ae8dd60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:32:41.093245 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6c4256-0339-41a8-acf0-1288df5eccca service neutron] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:32:41.093484 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6c4256-0339-41a8-acf0-1288df5eccca service neutron] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:32:41.093760 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6c4256-0339-41a8-acf0-1288df5eccca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:32:41.095473 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-4e6c4256-0339-41a8-acf0-1288df5eccca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:32:41.106048 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6c4256-0339-41a8-acf0-1288df5eccca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:32:41.107510 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6c4256-0339-41a8-acf0-1288df5eccca service neutron] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:32:41.116624 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1438/2891] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:32:41 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 01:32:45.350554 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=32 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:32:50.463599 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b9dd43-273b-4d4c-be2a-4b22e1f54fe3 service placement] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:32:50.472926 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b9dd43-273b-4d4c-be2a-4b22e1f54fe3 service placement] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:32:50.476450 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cfde87-0fb7-421f-8e04-88c45de46668 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0817c3320d5d41d8bba0550c8ae8dd60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:32:50.477081 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cfde87-0fb7-421f-8e04-88c45de46668 service neutron] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:32:50.477366 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cfde87-0fb7-421f-8e04-88c45de46668 service neutron] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:32:50.477640 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cfde87-0fb7-421f-8e04-88c45de46668 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:32:50.486377 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cfde87-0fb7-421f-8e04-88c45de46668 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:32:50.487803 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cfde87-0fb7-421f-8e04-88c45de46668 service neutron] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:32:50.497299 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1454/2892] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:32:50 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:32:51.112276 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=6 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:33:00.493107 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=6 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:33:16.120673 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6c4256-0339-41a8-acf0-1288df5eccca service neutron] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:33:16.129458 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-4e6c4256-0339-41a8-acf0-1288df5eccca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:33:16.187759 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6c4256-0339-41a8-acf0-1288df5eccca service neutron] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:33:16.192230 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42d611e-56d6-44e6-9a37-592fdf23f01c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:33:16.192899 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42d611e-56d6-44e6-9a37-592fdf23f01c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:33:16.193182 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42d611e-56d6-44e6-9a37-592fdf23f01c admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:33:16.193461 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42d611e-56d6-44e6-9a37-592fdf23f01c admin admin] PATH_INFO: `/v3/users/f3e944928454456cabf55a6871dd9cf3` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:33:16.194586 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42d611e-56d6-44e6-9a37-592fdf23f01c admin admin] RBAC: Authorizing `identity:delete_user(user_id=f3e944928454456cabf55a6871dd9cf3)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:33:16.197037 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42d611e-56d6-44e6-9a37-592fdf23f01c admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:33:16.373966 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-f42d611e-56d6-44e6-9a37-592fdf23f01c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f3e944928454456cabf55a6871dd9cf3'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:33:16.463463 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-f42d611e-56d6-44e6-9a37-592fdf23f01c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f3e944928454456cabf55a6871dd9cf3'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:33:16.464079 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-f42d611e-56d6-44e6-9a37-592fdf23f01c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f3e944928454456cabf55a6871dd9cf3'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:33:16.471204 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-f42d611e-56d6-44e6-9a37-592fdf23f01c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f3e944928454456cabf55a6871dd9cf3'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:33:16.471822 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-f42d611e-56d6-44e6-9a37-592fdf23f01c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f3e944928454456cabf55a6871dd9cf3'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:33:16.488677 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1439/2893] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 01:33:16 2022] DELETE /identity/v3/users/f3e944928454456cabf55a6871dd9cf3 => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:33:16.721699 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cfde87-0fb7-421f-8e04-88c45de46668 service neutron] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:33:16.732793 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cfde87-0fb7-421f-8e04-88c45de46668 service neutron] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:33:16.791180 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a431d6-1246-43e5-9bf2-8fb799477f78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:33:16.791811 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a431d6-1246-43e5-9bf2-8fb799477f78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:33:16.792101 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a431d6-1246-43e5-9bf2-8fb799477f78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:33:16.792378 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a431d6-1246-43e5-9bf2-8fb799477f78 admin admin] PATH_INFO: `/v3/projects/29c95fae9a51415b9245125f5faedb37` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:33:16.793533 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a431d6-1246-43e5-9bf2-8fb799477f78 admin admin] RBAC: Authorizing `identity:delete_project(project_id=29c95fae9a51415b9245125f5faedb37)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:33:16.795426 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a431d6-1246-43e5-9bf2-8fb799477f78 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:33:16.921226 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-33a431d6-1246-43e5-9bf2-8fb799477f78 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '29c95fae9a51415b9245125f5faedb37', 'request_id': 'req-33a431d6-1246-43e5-9bf2-8fb799477f78'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:33:16.926706 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-33a431d6-1246-43e5-9bf2-8fb799477f78 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '29c95fae9a51415b9245125f5faedb37', 'request_id': 'req-33a431d6-1246-43e5-9bf2-8fb799477f78'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:33:17.000191 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-33a431d6-1246-43e5-9bf2-8fb799477f78 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '29c95fae9a51415b9245125f5faedb37', 'request_id': 'req-33a431d6-1246-43e5-9bf2-8fb799477f78'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:33:17.011376 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-33a431d6-1246-43e5-9bf2-8fb799477f78 admin admin] The token cache is being invalidate because project 29c95fae9a51415b9245125f5faedb37 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81750) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 01:33:17.011796 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-33a431d6-1246-43e5-9bf2-8fb799477f78 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:33:17.012425 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-33a431d6-1246-43e5-9bf2-8fb799477f78 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:33:17.013911 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1455/2894] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 01:33:16 2022] DELETE /identity/v3/projects/29c95fae9a51415b9245125f5faedb37 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:33:17.019080 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc89f2f-c70e-4252-a08c-fc901119d78e None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:33:17.019373 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc89f2f-c70e-4252-a08c-fc901119d78e None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:33:17.019657 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc89f2f-c70e-4252-a08c-fc901119d78e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:33:17.039310 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-adc89f2f-c70e-4252-a08c-fc901119d78e None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:33:17.041087 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-adc89f2f-c70e-4252-a08c-fc901119d78e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:33:17.098910 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1440/2895] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:33:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:33:17.108190 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4173cb-b24c-467f-abd4-801353011df2 None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:33:17.108478 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4173cb-b24c-467f-abd4-801353011df2 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:33:17.108790 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4173cb-b24c-467f-abd4-801353011df2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:33:17.128454 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-ac4173cb-b24c-467f-abd4-801353011df2 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:33:17.129686 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-ac4173cb-b24c-467f-abd4-801353011df2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:33:17.137473 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1456/2896] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:33:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:33:17.141561 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc89f2f-c70e-4252-a08c-fc901119d78e None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:33:17.151011 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc89f2f-c70e-4252-a08c-fc901119d78e None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:33:17.155423 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f85b866-7b59-4c01-bb10-4ffcfc1e45dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:33:17.156037 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f85b866-7b59-4c01-bb10-4ffcfc1e45dd admin admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:33:17.156313 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f85b866-7b59-4c01-bb10-4ffcfc1e45dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:33:17.156586 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f85b866-7b59-4c01-bb10-4ffcfc1e45dd admin admin] PATH_INFO: `/v3/domains` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:33:17.157401 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f85b866-7b59-4c01-bb10-4ffcfc1e45dd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:33:17.157701 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f85b866-7b59-4c01-bb10-4ffcfc1e45dd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:33:17.159288 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f85b866-7b59-4c01-bb10-4ffcfc1e45dd admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:33:17.171085 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1441/2897] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Aug 31 01:33:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:33:17.175253 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4173cb-b24c-467f-abd4-801353011df2 None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:33:17.185193 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4173cb-b24c-467f-abd4-801353011df2 None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:33:17.189909 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a82013c-0632-460e-87d7-4bc91e9be615 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:33:17.190503 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a82013c-0632-460e-87d7-4bc91e9be615 admin admin] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:33:17.190786 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a82013c-0632-460e-87d7-4bc91e9be615 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:33:17.191064 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a82013c-0632-460e-87d7-4bc91e9be615 admin admin] PATH_INFO: `/v3/projects` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:33:17.191716 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a82013c-0632-460e-87d7-4bc91e9be615 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:33:17.193685 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a82013c-0632-460e-87d7-4bc91e9be615 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:33:17.281354 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1457/2898] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 01:33:17 2022] POST /identity/v3/projects => generated 379 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:33:17.285507 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f85b866-7b59-4c01-bb10-4ffcfc1e45dd admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:33:17.296109 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f85b866-7b59-4c01-bb10-4ffcfc1e45dd admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:33:17.299584 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:33:17.350699 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17dec7c-5b7a-432c-b6dd-c7770e9d917a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:33:17.351298 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17dec7c-5b7a-432c-b6dd-c7770e9d917a admin admin] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:33:17.351576 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17dec7c-5b7a-432c-b6dd-c7770e9d917a admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:33:17.351847 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17dec7c-5b7a-432c-b6dd-c7770e9d917a admin admin] PATH_INFO: `/v3/users` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:33:17.352559 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17dec7c-5b7a-432c-b6dd-c7770e9d917a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:33:17.355169 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17dec7c-5b7a-432c-b6dd-c7770e9d917a admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:33:17.456421 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1442/2899] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 01:33:17 2022] POST /identity/v3/users => generated 290 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:33:17.460597 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a82013c-0632-460e-87d7-4bc91e9be615 admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:33:17.471316 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a82013c-0632-460e-87d7-4bc91e9be615 admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:33:17.475861 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3618e945-ba07-4ad6-9b2a-521315199f54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:33:17.476448 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3618e945-ba07-4ad6-9b2a-521315199f54 admin admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:33:17.476746 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3618e945-ba07-4ad6-9b2a-521315199f54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:33:17.477027 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3618e945-ba07-4ad6-9b2a-521315199f54 admin admin] PATH_INFO: `/v3/roles` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:33:17.477657 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3618e945-ba07-4ad6-9b2a-521315199f54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:33:17.479895 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3618e945-ba07-4ad6-9b2a-521315199f54 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:33:17.489820 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1458/2900] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:33:17 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:33:17.493792 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17dec7c-5b7a-432c-b6dd-c7770e9d917a admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:33:17.503607 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17dec7c-5b7a-432c-b6dd-c7770e9d917a admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:33:17.508204 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802b1b1f-ce01-49aa-a64c-66e04848814b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:33:17.508826 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802b1b1f-ce01-49aa-a64c-66e04848814b admin admin] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:33:17.509115 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802b1b1f-ce01-49aa-a64c-66e04848814b admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:33:17.509395 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802b1b1f-ce01-49aa-a64c-66e04848814b admin admin] PATH_INFO: `/v3/roles` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:33:17.510033 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802b1b1f-ce01-49aa-a64c-66e04848814b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:33:17.511624 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802b1b1f-ce01-49aa-a64c-66e04848814b admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:33:17.522255 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1443/2901] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:33:17 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:33:17.526711 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3618e945-ba07-4ad6-9b2a-521315199f54 admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:33:17.536346 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3618e945-ba07-4ad6-9b2a-521315199f54 admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:33:17.541489 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6436df-085e-476a-afc2-f6fde1fba143 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:33:17.542120 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6436df-085e-476a-afc2-f6fde1fba143 admin admin] REQUEST_METHOD: `PUT` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:33:17.542396 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6436df-085e-476a-afc2-f6fde1fba143 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:33:17.542663 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6436df-085e-476a-afc2-f6fde1fba143 admin admin] PATH_INFO: `/v3/projects/8a6177d84e534b358b475672f474f6df/users/af97536937fb405198e1474fc134b493/roles/d0b9768f4d014766b9326387f2e11f34` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:33:17.556921 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6436df-085e-476a-afc2-f6fde1fba143 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a6177d84e534b358b475672f474f6df, user_id=af97536937fb405198e1474fc134b493, role_id=d0b9768f4d014766b9326387f2e11f34)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:33:17.559685 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6436df-085e-476a-afc2-f6fde1fba143 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:33:17.621851 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1459/2902] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 01:33:17 2022] PUT /identity/v3/projects/8a6177d84e534b358b475672f474f6df/users/af97536937fb405198e1474fc134b493/roles/d0b9768f4d014766b9326387f2e11f34 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:33:17.626935 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c649576-a351-48f6-9584-42934ff30037 None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:33:17.627231 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c649576-a351-48f6-9584-42934ff30037 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:33:17.627512 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c649576-a351-48f6-9584-42934ff30037 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:33:17.646815 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-7c649576-a351-48f6-9584-42934ff30037 None None] MFA Rules not processed for user `7772468996d2491aa2e6bc53000efa74`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:33:17.648009 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-7c649576-a351-48f6-9584-42934ff30037 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:33:17.707037 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1444/2903] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:33:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:33:17.717187 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6436df-085e-476a-afc2-f6fde1fba143 admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:33:17.718871 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-8f6436df-085e-476a-afc2-f6fde1fba143 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:33:17.738584 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:33:17.773981 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6436df-085e-476a-afc2-f6fde1fba143 admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:33:17.778226 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a64e89f-daff-426e-875a-a110fb693bf3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0817c3320d5d41d8bba0550c8ae8dd60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:33:17.778827 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a64e89f-daff-426e-875a-a110fb693bf3 service neutron] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:33:17.779108 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a64e89f-daff-426e-875a-a110fb693bf3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:33:17.779380 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a64e89f-daff-426e-875a-a110fb693bf3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:33:17.788627 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a64e89f-daff-426e-875a-a110fb693bf3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:33:17.790072 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a64e89f-daff-426e-875a-a110fb693bf3 service neutron] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:33:17.799264 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1460/2904] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:33:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:33:22.854555 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691b46d8-a1ab-4158-8d75-e430736b35c0 None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:33:22.854555 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691b46d8-a1ab-4158-8d75-e430736b35c0 None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:33:22.854555 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691b46d8-a1ab-4158-8d75-e430736b35c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:33:22.883017 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-691b46d8-a1ab-4158-8d75-e430736b35c0 None None] MFA Rules not processed for user `af97536937fb405198e1474fc134b493`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:33:22.884336 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-691b46d8-a1ab-4158-8d75-e430736b35c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:33:22.967647 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1445/2905] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:33:22 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:33:22.972279 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a64e89f-daff-426e-875a-a110fb693bf3 service neutron] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:33:22.982519 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a64e89f-daff-426e-875a-a110fb693bf3 service neutron] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:33:22.987365 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7869b9eb-4b37-4034-a6c6-100610698678 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:33:22.988004 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7869b9eb-4b37-4034-a6c6-100610698678 admin admin] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:33:22.988285 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7869b9eb-4b37-4034-a6c6-100610698678 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:33:22.988559 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7869b9eb-4b37-4034-a6c6-100610698678 admin admin] PATH_INFO: `/v3/projects` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:33:22.989234 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7869b9eb-4b37-4034-a6c6-100610698678 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:33:22.991133 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7869b9eb-4b37-4034-a6c6-100610698678 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:33:23.145469 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1461/2906] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 01:33:22 2022] POST /identity/v3/projects => generated 375 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:33:23.151200 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691b46d8-a1ab-4158-8d75-e430736b35c0 None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:33:23.162490 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691b46d8-a1ab-4158-8d75-e430736b35c0 None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:33:23.179592 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:33:23.228663 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626f59d7-70a3-4e63-a5e8-3fb703a8240c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:33:23.229321 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626f59d7-70a3-4e63-a5e8-3fb703a8240c admin admin] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:33:23.229604 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626f59d7-70a3-4e63-a5e8-3fb703a8240c admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:33:23.229882 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626f59d7-70a3-4e63-a5e8-3fb703a8240c admin admin] PATH_INFO: `/v3/users` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:33:23.230531 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626f59d7-70a3-4e63-a5e8-3fb703a8240c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:33:23.232449 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626f59d7-70a3-4e63-a5e8-3fb703a8240c admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:33:23.390147 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1446/2907] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 01:33:23 2022] POST /identity/v3/users => generated 294 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:33:23.394115 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7869b9eb-4b37-4034-a6c6-100610698678 admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:33:23.404463 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7869b9eb-4b37-4034-a6c6-100610698678 admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:33:23.414584 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9851f85d-0ab9-4271-a748-7287fe53c052 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:33:23.415251 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9851f85d-0ab9-4271-a748-7287fe53c052 admin admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:33:23.415538 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9851f85d-0ab9-4271-a748-7287fe53c052 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:33:23.415817 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9851f85d-0ab9-4271-a748-7287fe53c052 admin admin] PATH_INFO: `/v3/roles` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:33:23.416535 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9851f85d-0ab9-4271-a748-7287fe53c052 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:33:23.418186 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9851f85d-0ab9-4271-a748-7287fe53c052 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:33:23.432000 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1462/2908] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:33:23 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:33:23.436293 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626f59d7-70a3-4e63-a5e8-3fb703a8240c admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:33:23.447396 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626f59d7-70a3-4e63-a5e8-3fb703a8240c admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:33:23.452205 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c19d97-5f39-41c4-bf9c-cda2f69a1ea2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:33:23.453446 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c19d97-5f39-41c4-bf9c-cda2f69a1ea2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:33:23.453692 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c19d97-5f39-41c4-bf9c-cda2f69a1ea2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:33:23.454057 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c19d97-5f39-41c4-bf9c-cda2f69a1ea2 admin admin] PATH_INFO: `/v3/projects/2d46826da53d445fa93891f13fbe9826/users/de1cdad8024241069169ed1a5956475a/roles/d009411496b8452b88e77ffb8db015a7` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:33:23.471528 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c19d97-5f39-41c4-bf9c-cda2f69a1ea2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d46826da53d445fa93891f13fbe9826, user_id=de1cdad8024241069169ed1a5956475a, role_id=d009411496b8452b88e77ffb8db015a7)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:33:23.474340 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c19d97-5f39-41c4-bf9c-cda2f69a1ea2 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:33:23.543159 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1447/2909] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 01:33:23 2022] PUT /identity/v3/projects/2d46826da53d445fa93891f13fbe9826/users/de1cdad8024241069169ed1a5956475a/roles/d009411496b8452b88e77ffb8db015a7 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:33:23.547381 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9851f85d-0ab9-4271-a748-7287fe53c052 admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:33:23.558402 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9851f85d-0ab9-4271-a748-7287fe53c052 admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:33:23.615852 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae41f635-acaf-4c81-943f-5fd1bb7006a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:33:23.616557 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae41f635-acaf-4c81-943f-5fd1bb7006a3 admin admin] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:33:23.616941 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae41f635-acaf-4c81-943f-5fd1bb7006a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:33:23.617284 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae41f635-acaf-4c81-943f-5fd1bb7006a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:33:23.618005 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae41f635-acaf-4c81-943f-5fd1bb7006a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:33:23.620265 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae41f635-acaf-4c81-943f-5fd1bb7006a3 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:33:23.630620 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1463/2910] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 01:33:23 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 01:33:23.635492 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c19d97-5f39-41c4-bf9c-cda2f69a1ea2 admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:33:23.646899 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c19d97-5f39-41c4-bf9c-cda2f69a1ea2 admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:33:23.651651 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b157891-7532-4d3a-bf46-01973da9c1fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:33:23.652387 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b157891-7532-4d3a-bf46-01973da9c1fe admin admin] REQUEST_METHOD: `PUT` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:33:23.652783 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b157891-7532-4d3a-bf46-01973da9c1fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:33:23.653152 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b157891-7532-4d3a-bf46-01973da9c1fe admin admin] PATH_INFO: `/v3/projects/2d46826da53d445fa93891f13fbe9826/users/de1cdad8024241069169ed1a5956475a/roles/d009411496b8452b88e77ffb8db015a7` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:33:23.655098 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b157891-7532-4d3a-bf46-01973da9c1fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d46826da53d445fa93891f13fbe9826, user_id=de1cdad8024241069169ed1a5956475a, role_id=d009411496b8452b88e77ffb8db015a7)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:33:23.657817 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b157891-7532-4d3a-bf46-01973da9c1fe admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:33:23.669142 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1448/2911] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 01:33:23 2022] PUT /identity/v3/projects/2d46826da53d445fa93891f13fbe9826/users/de1cdad8024241069169ed1a5956475a/roles/d009411496b8452b88e77ffb8db015a7 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 01:33:28.862536 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce24049c-2fbf-49f3-ab48-fd19266d9d84 None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:33:28.863498 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce24049c-2fbf-49f3-ab48-fd19266d9d84 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:33:28.863498 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce24049c-2fbf-49f3-ab48-fd19266d9d84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:33:28.894356 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-ce24049c-2fbf-49f3-ab48-fd19266d9d84 None None] MFA Rules not processed for user `de1cdad8024241069169ed1a5956475a`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:33:28.895600 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-ce24049c-2fbf-49f3-ab48-fd19266d9d84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:33:28.970566 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1464/2912] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:33:28 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:33:28.987058 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b157891-7532-4d3a-bf46-01973da9c1fe admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:33:28.989017 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-5b157891-7532-4d3a-bf46-01973da9c1fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:33:29.068575 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b157891-7532-4d3a-bf46-01973da9c1fe admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:33:29.077053 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4609e7-e16e-4b95-a6b2-100e05f5df95 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8744c9de803048f583d0317d01f6a540', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:33:29.077681 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4609e7-e16e-4b95-a6b2-100e05f5df95 service nova] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:33:29.078019 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4609e7-e16e-4b95-a6b2-100e05f5df95 service nova] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:33:29.078296 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4609e7-e16e-4b95-a6b2-100e05f5df95 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:33:29.139978 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4609e7-e16e-4b95-a6b2-100e05f5df95 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:33:29.141618 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4609e7-e16e-4b95-a6b2-100e05f5df95 service nova] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:33:29.151097 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1449/2913] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:33:28 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:33:38.966902 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=95,keystone:INSERT=1 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:33:39.146550 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=93,keystone:INSERT=5 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:33:46.112290 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd690a35-fc46-42cb-9754-9c08518cb5c1 None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:33:46.112842 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd690a35-fc46-42cb-9754-9c08518cb5c1 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:33:46.112949 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd690a35-fc46-42cb-9754-9c08518cb5c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:33:46.132032 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-cd690a35-fc46-42cb-9754-9c08518cb5c1 None None] MFA Rules not processed for user `af97536937fb405198e1474fc134b493`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:33:46.133274 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-cd690a35-fc46-42cb-9754-9c08518cb5c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:33:46.140142 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1465/2914] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:33:46 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:33:46.454516 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4609e7-e16e-4b95-a6b2-100e05f5df95 service nova] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:33:46.507368 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4609e7-e16e-4b95-a6b2-100e05f5df95 service nova] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:33:46.510899 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf321f60-c923-4bf8-ab4b-59ced6a6b75b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0817c3320d5d41d8bba0550c8ae8dd60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:33:46.511493 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf321f60-c923-4bf8-ab4b-59ced6a6b75b service neutron] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:33:46.511773 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf321f60-c923-4bf8-ab4b-59ced6a6b75b service neutron] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:33:46.512046 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf321f60-c923-4bf8-ab4b-59ced6a6b75b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:33:46.513778 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-bf321f60-c923-4bf8-ab4b-59ced6a6b75b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:33:46.523041 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf321f60-c923-4bf8-ab4b-59ced6a6b75b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:33:46.524451 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf321f60-c923-4bf8-ab4b-59ced6a6b75b service neutron] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:33:46.532866 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1450/2915] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:33:46 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:33:51.009259 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd690a35-fc46-42cb-9754-9c08518cb5c1 None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:33:51.022329 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd690a35-fc46-42cb-9754-9c08518cb5c1 None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:33:51.028369 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493026b0-b5cd-4114-9655-70ff1a9e02ca service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8744c9de803048f583d0317d01f6a540', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:33:51.029029 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493026b0-b5cd-4114-9655-70ff1a9e02ca service nova] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:33:51.029316 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493026b0-b5cd-4114-9655-70ff1a9e02ca service nova] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:33:51.029593 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493026b0-b5cd-4114-9655-70ff1a9e02ca service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:33:51.031327 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-493026b0-b5cd-4114-9655-70ff1a9e02ca service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:33:51.044506 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493026b0-b5cd-4114-9655-70ff1a9e02ca service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:33:51.046129 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493026b0-b5cd-4114-9655-70ff1a9e02ca service nova] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:33:51.057333 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1466/2916] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:33:51 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:33:56.529937 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=26 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:34:00.739933 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf321f60-c923-4bf8-ab4b-59ced6a6b75b service neutron] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:34:00.741452 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-bf321f60-c923-4bf8-ab4b-59ced6a6b75b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:34:00.756880 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf321f60-c923-4bf8-ab4b-59ced6a6b75b service neutron] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:34:00.762217 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49814494-0d16-431a-b8f5-59bfcf6469c9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8744c9de803048f583d0317d01f6a540', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:34:00.762809 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49814494-0d16-431a-b8f5-59bfcf6469c9 service nova] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:34:00.763088 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49814494-0d16-431a-b8f5-59bfcf6469c9 service nova] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:34:00.763360 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49814494-0d16-431a-b8f5-59bfcf6469c9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:34:00.765069 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-49814494-0d16-431a-b8f5-59bfcf6469c9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:34:00.776541 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49814494-0d16-431a-b8f5-59bfcf6469c9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:34:00.778127 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49814494-0d16-431a-b8f5-59bfcf6469c9 service nova] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:34:00.788864 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1451/2917] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:34:00 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:34:01.057709 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=9 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:34:03.833941 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493026b0-b5cd-4114-9655-70ff1a9e02ca service nova] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:34:03.843813 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493026b0-b5cd-4114-9655-70ff1a9e02ca service nova] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:34:03.849941 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a57fff6-1ade-4c5c-9d17-b583ae8e3e5a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0817c3320d5d41d8bba0550c8ae8dd60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:34:03.849941 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a57fff6-1ade-4c5c-9d17-b583ae8e3e5a service neutron] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:34:03.849941 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a57fff6-1ade-4c5c-9d17-b583ae8e3e5a service neutron] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:34:03.849941 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a57fff6-1ade-4c5c-9d17-b583ae8e3e5a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:34:03.857120 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a57fff6-1ade-4c5c-9d17-b583ae8e3e5a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:34:03.858612 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a57fff6-1ade-4c5c-9d17-b583ae8e3e5a service neutron] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:34:03.867864 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1467/2918] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:34:03 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:34:05.739782 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49814494-0d16-431a-b8f5-59bfcf6469c9 service nova] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:34:05.751773 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49814494-0d16-431a-b8f5-59bfcf6469c9 service nova] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:34:05.757429 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab129863-22e6-4ac7-8597-25fd1dae0307 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8744c9de803048f583d0317d01f6a540', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:34:05.757973 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab129863-22e6-4ac7-8597-25fd1dae0307 service nova] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:34:05.758261 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab129863-22e6-4ac7-8597-25fd1dae0307 service nova] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:34:05.758541 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab129863-22e6-4ac7-8597-25fd1dae0307 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:34:05.760272 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-ab129863-22e6-4ac7-8597-25fd1dae0307 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:34:05.772945 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab129863-22e6-4ac7-8597-25fd1dae0307 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:34:05.774550 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab129863-22e6-4ac7-8597-25fd1dae0307 service nova] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:34:05.785783 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1452/2919] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:34:05 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:34:13.863710 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=6 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:34:15.780925 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=12 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:35:02.941588 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a57fff6-1ade-4c5c-9d17-b583ae8e3e5a service neutron] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:35:02.970158 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a57fff6-1ade-4c5c-9d17-b583ae8e3e5a service neutron] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:35:02.974495 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c732857-a294-4b0b-8916-c837fb3b6a5d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0817c3320d5d41d8bba0550c8ae8dd60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:35:02.975161 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c732857-a294-4b0b-8916-c837fb3b6a5d service neutron] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:35:02.975550 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c732857-a294-4b0b-8916-c837fb3b6a5d service neutron] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:35:02.975927 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c732857-a294-4b0b-8916-c837fb3b6a5d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:35:02.977867 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-2c732857-a294-4b0b-8916-c837fb3b6a5d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:35:02.988375 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c732857-a294-4b0b-8916-c837fb3b6a5d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:35:02.989988 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c732857-a294-4b0b-8916-c837fb3b6a5d service neutron] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:35:02.999182 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1468/2920] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:35:02 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 01:35:04.084369 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab129863-22e6-4ac7-8597-25fd1dae0307 service nova] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:35:04.086051 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-ab129863-22e6-4ac7-8597-25fd1dae0307 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:35:04.098481 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab129863-22e6-4ac7-8597-25fd1dae0307 service nova] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:35:04.102165 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682b0b71-ceda-45db-91c4-e2a96cb9d0e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0817c3320d5d41d8bba0550c8ae8dd60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:35:04.102782 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682b0b71-ceda-45db-91c4-e2a96cb9d0e5 service neutron] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:35:04.103068 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682b0b71-ceda-45db-91c4-e2a96cb9d0e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:35:04.103346 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682b0b71-ceda-45db-91c4-e2a96cb9d0e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:35:04.105111 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-682b0b71-ceda-45db-91c4-e2a96cb9d0e5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:35:04.119142 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682b0b71-ceda-45db-91c4-e2a96cb9d0e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:35:04.120631 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682b0b71-ceda-45db-91c4-e2a96cb9d0e5 service neutron] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:35:04.131678 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1453/2921] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:35:04 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:35:12.994986 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=6 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:35:14.126878 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=6 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:35:45.962825 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c732857-a294-4b0b-8916-c837fb3b6a5d service neutron] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:35:45.964230 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-2c732857-a294-4b0b-8916-c837fb3b6a5d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:35:46.031562 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c732857-a294-4b0b-8916-c837fb3b6a5d service neutron] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:35:46.036653 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c68f19-4cfe-438a-908b-a319b43547fb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dea0fe5b1d6b405cb47b95192580303b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:35:46.037282 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c68f19-4cfe-438a-908b-a319b43547fb service placement] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:35:46.037562 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c68f19-4cfe-438a-908b-a319b43547fb service placement] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:35:46.037836 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c68f19-4cfe-438a-908b-a319b43547fb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:35:46.039556 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-d5c68f19-4cfe-438a-908b-a319b43547fb service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:35:46.051830 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c68f19-4cfe-438a-908b-a319b43547fb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:35:46.053441 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c68f19-4cfe-438a-908b-a319b43547fb service placement] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:35:46.064120 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1469/2922] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:35:45 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:35:56.059169 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=32 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:36:04.735457 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b3c2fb-a144-4d86-abe2-9880f4dbf69b None None] REQUEST_METHOD: `POST` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:36:04.735946 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b3c2fb-a144-4d86-abe2-9880f4dbf69b None None] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:36:04.736061 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b3c2fb-a144-4d86-abe2-9880f4dbf69b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:36:04.755517 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.auth.core [None req-d8b3c2fb-a144-4d86-abe2-9880f4dbf69b None None] MFA Rules not processed for user `af97536937fb405198e1474fc134b493`. Rule list: `[]` (Enabled: `True`). {{(pid=81749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:36:04.756740 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-d8b3c2fb-a144-4d86-abe2-9880f4dbf69b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:36:04.763750 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1454/2923] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:36:04 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:36:14.749447 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=3 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:36:43.754337 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c68f19-4cfe-438a-908b-a319b43547fb service placement] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:36:43.755833 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-d5c68f19-4cfe-438a-908b-a319b43547fb service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:36:43.769862 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c68f19-4cfe-438a-908b-a319b43547fb service placement] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:36:43.780601 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c893db9-2dfe-4a14-a55a-d2f70ac2cf3f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dea0fe5b1d6b405cb47b95192580303b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:36:43.781323 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c893db9-2dfe-4a14-a55a-d2f70ac2cf3f service placement] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:36:43.781609 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c893db9-2dfe-4a14-a55a-d2f70ac2cf3f service placement] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:36:43.781888 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c893db9-2dfe-4a14-a55a-d2f70ac2cf3f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:36:43.783632 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-8c893db9-2dfe-4a14-a55a-d2f70ac2cf3f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:36:43.796286 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c893db9-2dfe-4a14-a55a-d2f70ac2cf3f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:36:43.797951 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c893db9-2dfe-4a14-a55a-d2f70ac2cf3f service placement] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:36:43.808294 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1470/2924] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:36:43 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:36:47.460745 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b3c2fb-a144-4d86-abe2-9880f4dbf69b None None] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:36:47.462255 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-d8b3c2fb-a144-4d86-abe2-9880f4dbf69b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:36:47.527128 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b3c2fb-a144-4d86-abe2-9880f4dbf69b None None] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:36:47.531441 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0d1553-7139-49cf-b87b-92e6e4607a2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:36:47.532060 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0d1553-7139-49cf-b87b-92e6e4607a2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:36:47.532338 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0d1553-7139-49cf-b87b-92e6e4607a2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:36:47.532637 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0d1553-7139-49cf-b87b-92e6e4607a2d admin admin] PATH_INFO: `/v3/users/181421ac2d7b49d496cabfd6f205f203` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:36:47.533765 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0d1553-7139-49cf-b87b-92e6e4607a2d admin admin] RBAC: Authorizing `identity:delete_user(user_id=181421ac2d7b49d496cabfd6f205f203)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:36:47.535662 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0d1553-7139-49cf-b87b-92e6e4607a2d admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:36:47.739900 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-4d0d1553-7139-49cf-b87b-92e6e4607a2d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '181421ac2d7b49d496cabfd6f205f203'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:36:47.801042 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-4d0d1553-7139-49cf-b87b-92e6e4607a2d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '181421ac2d7b49d496cabfd6f205f203'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:36:47.801606 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-4d0d1553-7139-49cf-b87b-92e6e4607a2d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '181421ac2d7b49d496cabfd6f205f203'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:36:47.808708 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-4d0d1553-7139-49cf-b87b-92e6e4607a2d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '181421ac2d7b49d496cabfd6f205f203'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:36:47.809370 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-4d0d1553-7139-49cf-b87b-92e6e4607a2d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '181421ac2d7b49d496cabfd6f205f203'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:36:47.826913 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1455/2925] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 01:36:47 2022] DELETE /identity/v3/users/181421ac2d7b49d496cabfd6f205f203 => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:36:48.161045 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c893db9-2dfe-4a14-a55a-d2f70ac2cf3f service placement] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:36:48.171899 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c893db9-2dfe-4a14-a55a-d2f70ac2cf3f service placement] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:36:48.231229 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8719f8df-0048-4625-a30a-95c57e8daa51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:36:48.231870 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8719f8df-0048-4625-a30a-95c57e8daa51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:36:48.232151 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8719f8df-0048-4625-a30a-95c57e8daa51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:36:48.232447 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8719f8df-0048-4625-a30a-95c57e8daa51 admin admin] PATH_INFO: `/v3/projects/970d06f2d1e24f0e9072a7f43bb4925b` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:36:48.233652 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8719f8df-0048-4625-a30a-95c57e8daa51 admin admin] RBAC: Authorizing `identity:delete_project(project_id=970d06f2d1e24f0e9072a7f43bb4925b)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:36:48.235544 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8719f8df-0048-4625-a30a-95c57e8daa51 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:36:48.345960 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-8719f8df-0048-4625-a30a-95c57e8daa51 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '970d06f2d1e24f0e9072a7f43bb4925b', 'request_id': 'req-8719f8df-0048-4625-a30a-95c57e8daa51'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:36:48.352441 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-8719f8df-0048-4625-a30a-95c57e8daa51 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '970d06f2d1e24f0e9072a7f43bb4925b', 'request_id': 'req-8719f8df-0048-4625-a30a-95c57e8daa51'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:36:48.424304 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-8719f8df-0048-4625-a30a-95c57e8daa51 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '970d06f2d1e24f0e9072a7f43bb4925b', 'request_id': 'req-8719f8df-0048-4625-a30a-95c57e8daa51'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:36:48.436663 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-8719f8df-0048-4625-a30a-95c57e8daa51 admin admin] The token cache is being invalidate because project 970d06f2d1e24f0e9072a7f43bb4925b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81750) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 01:36:48.437031 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-8719f8df-0048-4625-a30a-95c57e8daa51 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:36:48.437668 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-8719f8df-0048-4625-a30a-95c57e8daa51 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:36:48.439183 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1471/2926] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 01:36:48 2022] DELETE /identity/v3/projects/970d06f2d1e24f0e9072a7f43bb4925b => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:36:48.443245 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0d1553-7139-49cf-b87b-92e6e4607a2d admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:36:48.444640 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-4d0d1553-7139-49cf-b87b-92e6e4607a2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:36:48.457848 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0d1553-7139-49cf-b87b-92e6e4607a2d admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:36:48.514599 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f73fdf6-6e10-40f1-b86f-bfd4f7d41ac2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:36:48.515311 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f73fdf6-6e10-40f1-b86f-bfd4f7d41ac2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:36:48.515505 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f73fdf6-6e10-40f1-b86f-bfd4f7d41ac2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:36:48.515782 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f73fdf6-6e10-40f1-b86f-bfd4f7d41ac2 admin admin] PATH_INFO: `/v3/users/b6e35abdbb1f459492faf571236e1962` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:36:48.516941 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f73fdf6-6e10-40f1-b86f-bfd4f7d41ac2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b6e35abdbb1f459492faf571236e1962)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:36:48.518845 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f73fdf6-6e10-40f1-b86f-bfd4f7d41ac2 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:36:48.539967 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:36:48.718072 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-1f73fdf6-6e10-40f1-b86f-bfd4f7d41ac2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b6e35abdbb1f459492faf571236e1962'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:36:48.782535 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-1f73fdf6-6e10-40f1-b86f-bfd4f7d41ac2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b6e35abdbb1f459492faf571236e1962'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:36:48.783103 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-1f73fdf6-6e10-40f1-b86f-bfd4f7d41ac2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b6e35abdbb1f459492faf571236e1962'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:36:48.790771 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-1f73fdf6-6e10-40f1-b86f-bfd4f7d41ac2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b6e35abdbb1f459492faf571236e1962'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:36:48.791418 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-1f73fdf6-6e10-40f1-b86f-bfd4f7d41ac2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b6e35abdbb1f459492faf571236e1962'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:36:48.808317 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1456/2927] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 01:36:48 2022] DELETE /identity/v3/users/b6e35abdbb1f459492faf571236e1962 => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:36:49.058010 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8719f8df-0048-4625-a30a-95c57e8daa51 admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:36:49.061319 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-8719f8df-0048-4625-a30a-95c57e8daa51 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:36:49.076327 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8719f8df-0048-4625-a30a-95c57e8daa51 admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:36:49.134380 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58cff6e-5589-4e5d-975b-7840505acbe0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:36:49.135013 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58cff6e-5589-4e5d-975b-7840505acbe0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:36:49.135294 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58cff6e-5589-4e5d-975b-7840505acbe0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:36:49.135571 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58cff6e-5589-4e5d-975b-7840505acbe0 admin admin] PATH_INFO: `/v3/projects/dd9592e823ce4809b652aa7fd2029f49` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:36:49.136689 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58cff6e-5589-4e5d-975b-7840505acbe0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dd9592e823ce4809b652aa7fd2029f49)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:36:49.138610 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58cff6e-5589-4e5d-975b-7840505acbe0 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:36:49.160825 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:36:49.257989 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-b58cff6e-5589-4e5d-975b-7840505acbe0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dd9592e823ce4809b652aa7fd2029f49', 'request_id': 'req-b58cff6e-5589-4e5d-975b-7840505acbe0'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:36:49.263290 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-b58cff6e-5589-4e5d-975b-7840505acbe0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dd9592e823ce4809b652aa7fd2029f49', 'request_id': 'req-b58cff6e-5589-4e5d-975b-7840505acbe0'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:36:49.315653 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-b58cff6e-5589-4e5d-975b-7840505acbe0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dd9592e823ce4809b652aa7fd2029f49', 'request_id': 'req-b58cff6e-5589-4e5d-975b-7840505acbe0'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:36:49.327857 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-b58cff6e-5589-4e5d-975b-7840505acbe0 admin admin] The token cache is being invalidate because project dd9592e823ce4809b652aa7fd2029f49 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81750) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 01:36:49.328143 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-b58cff6e-5589-4e5d-975b-7840505acbe0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:36:49.328782 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-b58cff6e-5589-4e5d-975b-7840505acbe0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:36:49.330568 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1472/2928] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 01:36:49 2022] DELETE /identity/v3/projects/dd9592e823ce4809b652aa7fd2029f49 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:36:58.804799 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:DELETE=7,keystone:SELECT=20,keystone:INSERT=1 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:36:59.324396 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=16,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:37:54.945416 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f73fdf6-6e10-40f1-b86f-bfd4f7d41ac2 admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:37:54.946849 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-1f73fdf6-6e10-40f1-b86f-bfd4f7d41ac2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:37:55.003591 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f73fdf6-6e10-40f1-b86f-bfd4f7d41ac2 admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:37:55.050769 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde82076-5c0e-4333-ac98-f017ae1f3bb3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0817c3320d5d41d8bba0550c8ae8dd60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:37:55.051379 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde82076-5c0e-4333-ac98-f017ae1f3bb3 service neutron] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:37:55.051664 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde82076-5c0e-4333-ac98-f017ae1f3bb3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:37:55.051944 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde82076-5c0e-4333-ac98-f017ae1f3bb3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:37:55.053720 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-dde82076-5c0e-4333-ac98-f017ae1f3bb3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:37:55.063446 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde82076-5c0e-4333-ac98-f017ae1f3bb3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:37:55.064909 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde82076-5c0e-4333-ac98-f017ae1f3bb3 service neutron] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:37:55.073221 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1457/2929] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:37:54 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:37:58.897701 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8dec0b-c34d-41e2-9022-ba201273de90 None None] REQUEST_METHOD: `POST` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:37:58.898264 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8dec0b-c34d-41e2-9022-ba201273de90 None None] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:37:58.898861 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8dec0b-c34d-41e2-9022-ba201273de90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:37:58.917880 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.auth.core [None req-5b8dec0b-c34d-41e2-9022-ba201273de90 None None] MFA Rules not processed for user `af97536937fb405198e1474fc134b493`. Rule list: `[]` (Enabled: `True`). {{(pid=81750) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:37:58.919081 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-5b8dec0b-c34d-41e2-9022-ba201273de90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:37:58.973363 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1473/2930] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 01:37:58 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:37:59.013221 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde82076-5c0e-4333-ac98-f017ae1f3bb3 service neutron] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:37:59.014596 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-dde82076-5c0e-4333-ac98-f017ae1f3bb3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:37:59.070213 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde82076-5c0e-4333-ac98-f017ae1f3bb3 service neutron] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:37:59.073827 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b01d5e-a943-4527-bd73-37ad9ab3596a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5c8fc69c518412ba1aad61a9f4f558c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:37:59.074422 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b01d5e-a943-4527-bd73-37ad9ab3596a service cinder] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:37:59.074701 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b01d5e-a943-4527-bd73-37ad9ab3596a service cinder] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:37:59.074981 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b01d5e-a943-4527-bd73-37ad9ab3596a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:37:59.076661 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-10b01d5e-a943-4527-bd73-37ad9ab3596a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:37:59.146359 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b01d5e-a943-4527-bd73-37ad9ab3596a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:37:59.147837 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b01d5e-a943-4527-bd73-37ad9ab3596a service cinder] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:37:59.153573 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=100 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:37:59.159278 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1458/2931] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:37:59 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:37:59.440695 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8dec0b-c34d-41e2-9022-ba201273de90 None None] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:37:59.442115 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-5b8dec0b-c34d-41e2-9022-ba201273de90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:37:59.453724 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8dec0b-c34d-41e2-9022-ba201273de90 None None] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:37:59.457957 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecea5706-8f8f-4303-b08d-b2a3b4d9d747 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0817c3320d5d41d8bba0550c8ae8dd60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:37:59.458548 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecea5706-8f8f-4303-b08d-b2a3b4d9d747 service neutron] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:37:59.458826 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecea5706-8f8f-4303-b08d-b2a3b4d9d747 service neutron] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:37:59.459097 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecea5706-8f8f-4303-b08d-b2a3b4d9d747 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:37:59.460736 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-ecea5706-8f8f-4303-b08d-b2a3b4d9d747 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:37:59.469821 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecea5706-8f8f-4303-b08d-b2a3b4d9d747 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:37:59.471799 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecea5706-8f8f-4303-b08d-b2a3b4d9d747 service neutron] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:37:59.480066 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1474/2932] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:37:59 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:38:00.207856 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b01d5e-a943-4527-bd73-37ad9ab3596a service cinder] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:38:00.212985 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-10b01d5e-a943-4527-bd73-37ad9ab3596a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:38:00.217712 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecea5706-8f8f-4303-b08d-b2a3b4d9d747 service neutron] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:38:00.226829 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecea5706-8f8f-4303-b08d-b2a3b4d9d747 service neutron] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:38:00.230631 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55048f1-5f1c-4382-8162-670976f8f984 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0817c3320d5d41d8bba0550c8ae8dd60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:38:00.231239 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55048f1-5f1c-4382-8162-670976f8f984 service neutron] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:38:00.231520 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55048f1-5f1c-4382-8162-670976f8f984 service neutron] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:38:00.231802 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55048f1-5f1c-4382-8162-670976f8f984 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:38:00.233682 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-e55048f1-5f1c-4382-8162-670976f8f984 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:38:00.247917 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55048f1-5f1c-4382-8162-670976f8f984 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:38:00.249414 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55048f1-5f1c-4382-8162-670976f8f984 service neutron] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:38:00.261828 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1475/2933] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:38:00 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:38:00.283961 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b01d5e-a943-4527-bd73-37ad9ab3596a service cinder] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:38:00.288880 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55056c17-178a-43ac-92b7-84adf2ee785f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dea0fe5b1d6b405cb47b95192580303b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:38:00.289659 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55056c17-178a-43ac-92b7-84adf2ee785f service placement] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:38:00.290078 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55056c17-178a-43ac-92b7-84adf2ee785f service placement] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:38:00.290397 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55056c17-178a-43ac-92b7-84adf2ee785f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:38:00.292116 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-55056c17-178a-43ac-92b7-84adf2ee785f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:38:00.304574 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55056c17-178a-43ac-92b7-84adf2ee785f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:38:00.306194 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55056c17-178a-43ac-92b7-84adf2ee785f service placement] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:38:00.316189 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1459/2934] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:38:00 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:38:10.257448 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=37 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:38:10.311779 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=32 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:39:46.065010 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55048f1-5f1c-4382-8162-670976f8f984 service neutron] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:39:46.074297 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55048f1-5f1c-4382-8162-670976f8f984 service neutron] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:39:46.078073 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e9f6d0-1178-438d-aa51-8f5d028ef7af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0817c3320d5d41d8bba0550c8ae8dd60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:39:46.078678 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e9f6d0-1178-438d-aa51-8f5d028ef7af service neutron] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:39:46.078958 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e9f6d0-1178-438d-aa51-8f5d028ef7af service neutron] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:39:46.079229 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e9f6d0-1178-438d-aa51-8f5d028ef7af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:39:46.081259 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-72e9f6d0-1178-438d-aa51-8f5d028ef7af service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:39:46.092511 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e9f6d0-1178-438d-aa51-8f5d028ef7af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:39:46.093958 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e9f6d0-1178-438d-aa51-8f5d028ef7af service neutron] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:39:46.102618 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1476/2935] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:39:46 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:39:47.058914 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55056c17-178a-43ac-92b7-84adf2ee785f service placement] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:39:47.060512 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-55056c17-178a-43ac-92b7-84adf2ee785f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:39:47.078404 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55056c17-178a-43ac-92b7-84adf2ee785f service placement] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:39:47.086714 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1901c362-36b5-4963-9fef-c4b5f65521b9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8744c9de803048f583d0317d01f6a540', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:39:47.088577 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1901c362-36b5-4963-9fef-c4b5f65521b9 service nova] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:39:47.088935 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1901c362-36b5-4963-9fef-c4b5f65521b9 service nova] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:39:47.089220 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1901c362-36b5-4963-9fef-c4b5f65521b9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:39:47.090974 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-1901c362-36b5-4963-9fef-c4b5f65521b9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:39:47.104512 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1901c362-36b5-4963-9fef-c4b5f65521b9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:39:47.106113 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1901c362-36b5-4963-9fef-c4b5f65521b9 service nova] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:39:47.118314 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1460/2936] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:39:47 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:39:56.098670 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=6 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:39:57.114190 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=6 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:40:00.547392 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e9f6d0-1178-438d-aa51-8f5d028ef7af service neutron] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:40:00.548862 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-72e9f6d0-1178-438d-aa51-8f5d028ef7af service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:40:00.560399 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e9f6d0-1178-438d-aa51-8f5d028ef7af service neutron] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:40:00.563953 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2057f27e-ea8c-487b-a84b-509fe7f28cfb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0817c3320d5d41d8bba0550c8ae8dd60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:40:00.564545 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2057f27e-ea8c-487b-a84b-509fe7f28cfb service neutron] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:40:00.564862 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2057f27e-ea8c-487b-a84b-509fe7f28cfb service neutron] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:40:00.565139 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2057f27e-ea8c-487b-a84b-509fe7f28cfb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:40:00.566767 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-2057f27e-ea8c-487b-a84b-509fe7f28cfb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:40:00.628823 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2057f27e-ea8c-487b-a84b-509fe7f28cfb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:40:00.630255 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2057f27e-ea8c-487b-a84b-509fe7f28cfb service neutron] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:40:00.640355 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1477/2937] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:40:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:40:03.886114 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1901c362-36b5-4963-9fef-c4b5f65521b9 service nova] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:40:03.911229 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1901c362-36b5-4963-9fef-c4b5f65521b9 service nova] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:40:03.917625 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64afbdfe-3efa-421b-95e6-8ef28baff6f5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dea0fe5b1d6b405cb47b95192580303b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:40:03.918245 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64afbdfe-3efa-421b-95e6-8ef28baff6f5 service placement] REQUEST_METHOD: `GET` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:40:03.918537 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64afbdfe-3efa-421b-95e6-8ef28baff6f5 service placement] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:40:03.918816 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64afbdfe-3efa-421b-95e6-8ef28baff6f5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:40:03.920892 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-64afbdfe-3efa-421b-95e6-8ef28baff6f5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:40:03.933749 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64afbdfe-3efa-421b-95e6-8ef28baff6f5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:40:03.935329 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64afbdfe-3efa-421b-95e6-8ef28baff6f5 service placement] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:40:03.945488 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1461/2938] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:40:03 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:40:07.604814 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2057f27e-ea8c-487b-a84b-509fe7f28cfb service neutron] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:40:07.606389 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-2057f27e-ea8c-487b-a84b-509fe7f28cfb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:40:07.618585 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2057f27e-ea8c-487b-a84b-509fe7f28cfb service neutron] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:40:07.623277 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b390d3aa-442c-4fe5-9f91-408f1dca7cf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:40:07.623931 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b390d3aa-442c-4fe5-9f91-408f1dca7cf0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:40:07.624215 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b390d3aa-442c-4fe5-9f91-408f1dca7cf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:40:07.624492 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b390d3aa-442c-4fe5-9f91-408f1dca7cf0 admin admin] PATH_INFO: `/v3/users/af97536937fb405198e1474fc134b493` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:40:07.625640 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b390d3aa-442c-4fe5-9f91-408f1dca7cf0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=af97536937fb405198e1474fc134b493)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:40:07.627553 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b390d3aa-442c-4fe5-9f91-408f1dca7cf0 admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:40:07.781848 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-b390d3aa-442c-4fe5-9f91-408f1dca7cf0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af97536937fb405198e1474fc134b493'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:40:07.793691 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-b390d3aa-442c-4fe5-9f91-408f1dca7cf0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af97536937fb405198e1474fc134b493'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:40:07.794358 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-b390d3aa-442c-4fe5-9f91-408f1dca7cf0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af97536937fb405198e1474fc134b493'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:40:07.801162 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-b390d3aa-442c-4fe5-9f91-408f1dca7cf0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af97536937fb405198e1474fc134b493'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:40:07.887805 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-b390d3aa-442c-4fe5-9f91-408f1dca7cf0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af97536937fb405198e1474fc134b493'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:40:07.894487 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1478/2939] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 01:40:07 2022] DELETE /identity/v3/users/af97536937fb405198e1474fc134b493 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:40:08.127239 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64afbdfe-3efa-421b-95e6-8ef28baff6f5 service placement] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:40:08.139643 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64afbdfe-3efa-421b-95e6-8ef28baff6f5 service placement] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:40:08.198967 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf9947e-65ed-4a29-9fa2-27e78fc72ad4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:40:08.199600 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf9947e-65ed-4a29-9fa2-27e78fc72ad4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:40:08.199886 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf9947e-65ed-4a29-9fa2-27e78fc72ad4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:40:08.200164 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf9947e-65ed-4a29-9fa2-27e78fc72ad4 admin admin] PATH_INFO: `/v3/projects/8a6177d84e534b358b475672f474f6df` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:40:08.201325 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf9947e-65ed-4a29-9fa2-27e78fc72ad4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8a6177d84e534b358b475672f474f6df)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:40:08.203396 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf9947e-65ed-4a29-9fa2-27e78fc72ad4 admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:40:08.297836 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-bbf9947e-65ed-4a29-9fa2-27e78fc72ad4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8a6177d84e534b358b475672f474f6df', 'request_id': 'req-bbf9947e-65ed-4a29-9fa2-27e78fc72ad4'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:40:08.304113 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-bbf9947e-65ed-4a29-9fa2-27e78fc72ad4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8a6177d84e534b358b475672f474f6df', 'request_id': 'req-bbf9947e-65ed-4a29-9fa2-27e78fc72ad4'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:40:08.310570 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-bbf9947e-65ed-4a29-9fa2-27e78fc72ad4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8a6177d84e534b358b475672f474f6df', 'request_id': 'req-bbf9947e-65ed-4a29-9fa2-27e78fc72ad4'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:40:08.382359 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-bbf9947e-65ed-4a29-9fa2-27e78fc72ad4 admin admin] The token cache is being invalidate because project 8a6177d84e534b358b475672f474f6df was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81749) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 01:40:08.382655 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-bbf9947e-65ed-4a29-9fa2-27e78fc72ad4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:40:08.383310 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-bbf9947e-65ed-4a29-9fa2-27e78fc72ad4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:40:08.384826 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1462/2940] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 01:40:08 2022] DELETE /identity/v3/projects/8a6177d84e534b358b475672f474f6df => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:40:08.388972 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b390d3aa-442c-4fe5-9f91-408f1dca7cf0 admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:40:08.390434 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-b390d3aa-442c-4fe5-9f91-408f1dca7cf0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:40:08.403984 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b390d3aa-442c-4fe5-9f91-408f1dca7cf0 admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:40:08.460195 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46e0e14-7359-4f74-931d-441f2322c50c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:40:08.460859 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46e0e14-7359-4f74-931d-441f2322c50c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:40:08.461197 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46e0e14-7359-4f74-931d-441f2322c50c admin admin] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:40:08.461418 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46e0e14-7359-4f74-931d-441f2322c50c admin admin] PATH_INFO: `/v3/users/de1cdad8024241069169ed1a5956475a` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:40:08.462553 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46e0e14-7359-4f74-931d-441f2322c50c admin admin] RBAC: Authorizing `identity:delete_user(user_id=de1cdad8024241069169ed1a5956475a)` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:40:08.464469 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46e0e14-7359-4f74-931d-441f2322c50c admin admin] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:40:08.469719 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:40:08.661650 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-a46e0e14-7359-4f74-931d-441f2322c50c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de1cdad8024241069169ed1a5956475a'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:40:08.673691 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-a46e0e14-7359-4f74-931d-441f2322c50c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de1cdad8024241069169ed1a5956475a'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:40:08.674358 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-a46e0e14-7359-4f74-931d-441f2322c50c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de1cdad8024241069169ed1a5956475a'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:40:08.681712 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-a46e0e14-7359-4f74-931d-441f2322c50c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de1cdad8024241069169ed1a5956475a'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:40:08.756717 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.notifications [None req-a46e0e14-7359-4f74-931d-441f2322c50c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de1cdad8024241069169ed1a5956475a'} {{(pid=81750) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:40:08.762885 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1479/2941] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 01:40:08 2022] DELETE /identity/v3/users/de1cdad8024241069169ed1a5956475a => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:40:09.006685 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf9947e-65ed-4a29-9fa2-27e78fc72ad4 admin admin] Authenticating user token {{(pid=81749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:40:09.008166 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.fernet_utils [None req-bbf9947e-65ed-4a29-9fa2-27e78fc72ad4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:40:09.023115 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf9947e-65ed-4a29-9fa2-27e78fc72ad4 admin admin] Validating token access rules against request {{(pid=81749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:40:09.079276 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1312e2b7-9a22-4816-bea5-701a2ebbb7fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7772468996d2491aa2e6bc53000efa74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77c8e2846a11431eb786ad3a5192ea92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:40:09.079900 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1312e2b7-9a22-4816-bea5-701a2ebbb7fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:40:09.080179 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1312e2b7-9a22-4816-bea5-701a2ebbb7fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:40:09.080455 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1312e2b7-9a22-4816-bea5-701a2ebbb7fa admin admin] PATH_INFO: `/v3/projects/2d46826da53d445fa93891f13fbe9826` {{(pid=81749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:40:09.081917 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1312e2b7-9a22-4816-bea5-701a2ebbb7fa admin admin] RBAC: Authorizing `identity:delete_project(project_id=2d46826da53d445fa93891f13fbe9826)` {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:40:09.083924 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1312e2b7-9a22-4816-bea5-701a2ebbb7fa admin admin] RBAC: Authorization granted {{(pid=81749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:40:09.104825 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:40:09.222490 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-1312e2b7-9a22-4816-bea5-701a2ebbb7fa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2d46826da53d445fa93891f13fbe9826', 'request_id': 'req-1312e2b7-9a22-4816-bea5-701a2ebbb7fa'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:40:09.227983 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-1312e2b7-9a22-4816-bea5-701a2ebbb7fa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2d46826da53d445fa93891f13fbe9826', 'request_id': 'req-1312e2b7-9a22-4816-bea5-701a2ebbb7fa'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:40:09.234152 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-1312e2b7-9a22-4816-bea5-701a2ebbb7fa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2d46826da53d445fa93891f13fbe9826', 'request_id': 'req-1312e2b7-9a22-4816-bea5-701a2ebbb7fa'} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:40:09.318896 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-1312e2b7-9a22-4816-bea5-701a2ebbb7fa admin admin] The token cache is being invalidate because project 2d46826da53d445fa93891f13fbe9826 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81749) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 01:40:09.319713 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-1312e2b7-9a22-4816-bea5-701a2ebbb7fa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:40:09.320440 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG keystone.notifications [None req-1312e2b7-9a22-4816-bea5-701a2ebbb7fa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:40:09.321997 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: [pid: 81749|app: 0|req: 1463/2942] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 01:40:09 2022] DELETE /identity/v3/projects/2d46826da53d445fa93891f13fbe9826 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:40:18.785411 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=23,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:40:19.377324 dv-cinder-855288-2-iscsi devstack@keystone.service[81749]: DEBUG dbcounter [-] [81749] Writing DB stats keystone:SELECT=16,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:40:49.464199 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46e0e14-7359-4f74-931d-441f2322c50c admin admin] Authenticating user token {{(pid=81750) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:40:49.465629 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-a46e0e14-7359-4f74-931d-441f2322c50c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:40:49.599867 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46e0e14-7359-4f74-931d-441f2322c50c admin admin] Validating token access rules against request {{(pid=81750) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:40:49.655546 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c783d513-22ad-4e54-b96e-fc18a83c249d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dea0fe5b1d6b405cb47b95192580303b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60b2abc15c2849ba8a1ed92117cc981b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81750) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:40:49.656149 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c783d513-22ad-4e54-b96e-fc18a83c249d service placement] REQUEST_METHOD: `GET` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:40:49.656428 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c783d513-22ad-4e54-b96e-fc18a83c249d service placement] SCRIPT_NAME: `/identity` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:40:49.656702 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c783d513-22ad-4e54-b96e-fc18a83c249d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81750) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:40:49.658403 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.fernet_utils [None req-c783d513-22ad-4e54-b96e-fc18a83c249d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81750) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:40:49.671258 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c783d513-22ad-4e54-b96e-fc18a83c249d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:40:49.672852 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c783d513-22ad-4e54-b96e-fc18a83c249d service placement] RBAC: Authorization granted {{(pid=81750) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:40:49.682595 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: [pid: 81750|app: 0|req: 1480/2943] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 01:40:49 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:40:59.678749 dv-cinder-855288-2-iscsi devstack@keystone.service[81750]: DEBUG dbcounter [-] [81750] Writing DB stats keystone:SELECT=89 {{(pid=81750) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}